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PREFACE 


This  paper  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  for  the  Office 
of  the  Assistant  Secretary  of  Defense  (Force  Management  and  Personnel)  {OASD(FM&P)] 
under  contract  MDA  903  89  C  0003,  Task  Order  T-L7-798,  Manpower  Issues,  issued  15 
March  1990.  The  objective  of  the  task  was  to  identify  promising  approaches  to  maintaining 
strong  military  manpower  capability  during  a  period  of  declining  budgets  and  force  levels. 
This  is  one  of  a  total  of  seven  papers  to  be  published.  Each  of  the  seven  papers  covers  a 
specific  area  of  military  manpower  management:  the  proper  experience  mix,  personnel 
movement,  the  timing  of  training,  lateral  entry,  the  link  between  career  progression  and 
assumption  of  management  responsibilities,  individual  training  methods,  and  increased  use 
of  simulators  for  training.  The  topic  of  this  paper  is  individual  training  methods. 

This  work  was  reviewed  by  Waynard  C.  Devers  and  William  T.  Mayfield  of  IDA 
and  by  Harry  J.  Gilman,  an  IDA  consultant 
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1.  INTRODUCTION  AND  SUMMARY 


A.  INTRODUCTION 

At  any  given  time,  one  of  every  seven  persons  in  the  active  military  is  receiving, 
giving,  or  supporting  individual  schoolhouse  training.  In  FY  1989,  6.7%  of  the  total 
Department  of  Defense  (DoD)  budget  was  spent  on  formal  individual  training.  Therefore, 
in  an  era  of  declining  personnel  ceilings  and  declining  budgets,  it  seems  appropriate  to 
examine  individual  training  for  potential  personnel  and  budget  savings. 

Over  the  past  twenty  years,  a  wide  variety  of  new  technologies  for  delivery  of 
training  have  been  developed.  Tltese  technologies  are  included  under  the  general  category 
of  Interactive  Courseware  (ICW),  and  include  approaches  such  as  computer- assisted 
instruction,  computer-managed  instruction,  and  interactive  videodisc  instruction.  The 
development  of  these  technologies  has  been  made  affordable  in  large  part  because  of  the 
rapid  decline  in  the  price  of  digital  electronics. 

The  main  hypothesis  to  be  examined  hene  is  that  these  ICW  technologies  offer  a 
cost-effective  way  to  deliver  instruction.  We  examine  evidence  that  ICW  will  diminish  the 
amount  of  time  students  spend  in  school  and  decrease  the  number  of  instructors  needed  to 
deliver  instruction.  If  this  is  the  case,  we  would  accept  the  likelihood  of  a  decrease  in  the 
support  personnel  requirements  for  training.  We  also  need  to  see  if  these  decreases  can  be 
achieved  at  a  cost  less  than  the  student,  insti-uctor,  and  support  pay  and  allowances  saved. 

An  additional  issue  to  be  discussed  is  whether  widespread  insertion  of  ICW  training 
delivery  approaches  would  require  a  reexamination  of  the  way  we  determine  training 
requirements. 

B.  SUMMARY 

1 .  Quantitative  Results 

The  data  presented  support  the  hypotheses  that  money  and  personnel  time  would  be 
saved  by  more  extensive  use  of  ICW.  The  evidence  is  sparse,  but  ICW  does  not  appear  to 
be  widely  used  in  military  training.  A  rough  estimate  of  the  effect  of  applying  these 
technologies  widely  in  individual,  specialized-skill  training  alone  suggests  DoD-wide 
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annual  cost  savings  of  $130  million  to  $160  million.  These  savings  occur  if  courses  that 
instruct  20-25%  of  annual  student  throughput  are  converted  to  ICW.  The  main  source  of 
these  savings  is  from  a  decrease  of  20-25%  in  the  average  time  students  are  under 
instruction  using  ICW  compared  with  conventional  methods.  These  changes  would  result 
in  9-1 1  thousand  fewer  students  in  training  at  any  given  time  with  no  change  in  the  number 
of  graduates.  The  development  and  maintenance  costs  for  these  ICW  technologies  seem  to 
be  equal  to  or  less  than  conventional  instructional  approaches, 

2 .  Qualitative  Issues 

Given  the  forecast  price  declines  in  the  hardware  and  software  used  for  ICW,  there 
is  every  reason  to  assume  its  relative  cost  will  fall  further.  In  addition,  instructional 
delivery  methods  that  can  be  more  decentralized  and  available  on  student  demand  may 
become  more  important  due  to  possible  changes  in  total  force  policy,  such  as  those  that 
increase  the  use  of  Reserve  components. 

Many  explanations  are  offered  for  lack  of  ICW  use.  One  possibility  is  that  the  costs 
avoided  for  individual  course  components  may  not  translate  into  course  sequence  time 
savings.  ICW  may  be  under-used  because  training  policymakers  are  not  aware  of  the  full 
costs  of  training  or  because  training  managers  are  not  given  incentives  to  manage  based  on 
those  costs.  In  addition,  lock-step  instruction  may  also  provide  non-training  benefits,  such 
as  team-building,  that  are  difficult  to  quantify. 

DoD  directives  and  instructions  exist  that  encourage  the  development  and  use  of 
innovative  technologies.  A  recently  enacted  Instruction  develops  procedures  to  make  it  less 
costly  to:  maintain  and  enhance  ICW  material,  find  out  about  its  existence,  and  disseminate 
it  to  interested  parties. 

3 .  Recommendations 

Because  ICW  training  appears  to  be  cost-effective,  efforts  should  be  made  to 
develop  policy  direction  that  would  make  consideration  of  such  technologies  more  common 
during  major  revisions  of  instructional  design  and  delivery  methods. 

Increased  use  of  ICW  will  tend  to  increase  the  amount  of  individualization, 
including  self-pacing,  in  training.  This  has  the  potential  to  change  the  way  students  learn 
and  the  amount  of  knowledge  they  accumulate.  If  these  changes  are  significant,  they  may 
change  the  amount  and  kinds  of  knowledge  and  skills  we  want  to  provide  through  training. 
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C.  REPORT  OUTLINE 


The  remainder  of  this  report  contains  four  main  sections.  The  next  section  (Section 
H)  describes  the  training  establishment  and  reviews  current  policies  on  training  and  current 
training  methods.  Section  III  summarizes  the  research  literature  on  potential  cost  and  time 
savings  of  ICW,  and  presents  a  first-order  estimate  of  the  cost  savings  that  might  result 
from  wider  use  of  ICW  in  military  training.  Section  IV  discusses  and  analyzes  some  of  the 
difficulties  with  and  uncertainties  about  implementing  a  policy  on  ICW.  It  also  suggests 
why  we  might  want  to  reexamine  our  approach  to  training  requirements  determination. 
Section  V  discusses  possible  policy  changes  and  further  analysis  that  might  be  undertaken. 
Appendix  A  describes  a  more  detailed  analysis  of  Navy  data. 
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II.  THE  INDIVIDUAL  TRAINING 
ESTABLISHMENT,  POLICIES,  AND  METHODS 


This  section  provides  information  on  the  size  of  the  individual  training  and 
education  establishment  and  the  training  load  that  it  supports  (most  of  the  measures  of  size 
are  in  dollars  rather  than  numbers  of  personnel).  A  brief  description  of  the  DoD  directives 
and  instructions  that  guide  training  policy  and  a  discussion  of  the  instructional  delivery 
methods  currently  being  used  are  included. 

A.  THE  INDIVroUAL  TRAINING  AND  EDUCATION  ESTABLISHMENT 

Data  on  DoD  individual  training  and  education  are  reported  aimually  in  the  "Military 
Manpower  Training  Report"  (MMTR)  [1],  which  is  the  source  of  most  of  the  data  in  this 
section. 

In  FY  1989,  6  7%  of  the  DoD  budget  was  spent  on  individual  training.'  At  any 
one  time,  approximately  8.6%  of  the  active  force  was  in  training,  4.2%  was  engaged  in 
delivering  or  directly  supporting  training  delivery,  and  another  1.4%  was  allocated  to  Base 
Operating  Support  (BOS)  of  the  training  function.  In  summary,  on  an  average  day  in 
1989,  one  person  of  every  seven  in  the  active  military  was  either  receiving,  giving,  or 
supporting  individual  training.  Also,  2%  of  the  DoD  civilian  workforce  '  'as  engaged  in 
training  and  direct  training  support  and  3.7%  was  engaged  in  BOS  for  the  training 
function.  Finally,  3.1%  of  the  reserves  are  in  training.^ 

Selected  FY  1989  data  are  shown  in  Tables  1, 2,  and  3.  The  dollar  figures  for  all 
individual  training  accumulated  in  Subtotal  1  (Table  1)  include  student  pay  and  allowances 
as  well  as  all  other  training  costs.  Student  and  instructor  loads,  sometimes  called  Average- 
on-Board  (AOB),  have  been  relatively  constant  over  the  last  ten  years  (±  5%),  and  the 
financial  resources  are  programmed  to  be  of  similar  constant  dollar  magnitudes  for 
FY  1990  and  FY  1991. 


'  See  Reference  [2],  p.  157,  SulHotal,  DoI>— Military,  $290.8  billion. 

2  According  to  Reference  [3],  cnd*FY  1989  active  fwce  was  2.131  million;  Reserves.  1.171  million;  and 
civilians,  1.037  millitm. 
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Tabt«  1.  Funding  of  Individual  Training 
by  Training  Catagory,  FY  1989 


TabI# 


Training  Categorv 

MlllHMli 

of 

Spcaalucd'Skili  Tnuiung 

$6,015  2 

FUgbi  Training 

2.375  4 

Recnui  Training 

1.7  Y0  2 

Anny  One  Siaiion  Unit  Training 

5‘il  1 

Mcdkai  Training 

Ml  4 

PnofmioaaJ  UcvcJopnnent  Educatioo 

725  0 

OflTiccr  Acquisition  Training 

5305 

Subioui  t 

1 2.KH<J  0 

BOS  and  Diroct  Training  Suppon 

4.405  4 

TDY  Com  for  Traming 

640  0 

PCS  Com  for  Training 

370  7 

ManagemoH  Headquanm 

155  6 

Suixout  2 

5.57!  7 

Reserve  Cocnpooeni  Pay  and  Altowances 

^6 

Subtotal  3 

940  6 

Total 

19.401  .3 

Source  Reference  (I),  p  IX -4  to  DC 

No<e<;  BOS  «  Bim  Opcntifl|  Support.  TDY  »  Tcsiporery’ 

rXity;  end  PCS  »  PerotMcai  Cbesge  of  Surtjoo 


labia  2.  Activa  and  Raaarva  Training  Loads 


Tbotrandit  of  Pmonnei 
Active  183.R 

Reserve  «x]  Guard  36.3 

Total _ 220.2 _ 

Source:  Reference  { 1 ).  n  o 

Note:  TouJ  doe*  not  edd  doe  to  rouodioi 


DoD  Manpowar  In  Support  of  Individual  Training.  FY  1989 


illEHHIIIi 

1 

1 

MiliUffy 

Civilian 

Total 

Direr 

89.2 

21.0 

110.2 

BOS  Sid  HQ 

30.1 

38.4 

68.5 

Total 

119.4 

59.3 

178.7 

Source;  Referaace  (t],  p.  Vin-3  and  V-ni-4 
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However,  the  MMTR  understates  the  magnitude  of  the  total  military  training 
enterprise.  It  only  concerns  instruction  conducted  in  formal  courses  by  organizations 
whose  primary  mission  is  education  or  training.  It  excludes  job-site  training,  factory  and 
unit  training  for  new  systems,  organized  team  trauning  for  performance  of  specific  military 
missions,  and  all  field  exercises.  The  magnitude  of  total  resources  allocated  to  training  is 
difficult  to  determine,  but  it  is  likely  to  exceed  the  MMTR  dollar  projections  by  a  factor  of 
two  or  three. 

B .  CURRENT  POLICIES 

This  section  consists  of  three  subsections.  The  first  subsection  discusses  the 
system  activities  approach  to  analyzing  existing  training.  The  second  subsection  briefly 
summarizes  the  two  main  training  DoD  directives  and  one  instruction  of  interest.  The  third 
subsection  discusses  how  these  policies  affect  choices  of  instructional  media.  Together, 
these  three  subsections  provide  an  analytical  approach  to  conceiving  training  systems, 
discuss  the  procedures  that  have  been  codified  to  implement  this  analytical  approach,  and 
briefly  describe  the  effects  of  the  approach  and  procedures  on  the  development  of 
instructional  media  for  UoD  training. 

1 .  System  Activities 

System  activities  consist  of  analysis,  design,  production,  delivery,  and  evaluation. 
Analysis  in  training  primarily  concerns  what  to  train  and  what  objectives  to  seek.  It  takes 
military  missions  and  the  jobs  required  to  perfewm  them  as  input  and  produces  the  learning 
and  production  objectives  specified  for  a  training  system.  Design  establishes  the 
instructional  tactics  to  be  used  by  a  training  system.  Design  takes  as  input  the  learning  and 
production  objectives  of  a  training  system  and  produces  specifications  for  developing  the 
media  and  training  content,  or  training  materials,  that  are  intended  to  accomplish  these 
objectives  in  the  most  cost-effective  manner.  Production  takes  as  input  the  training  system 
materials  specifications  established  by  the  design  process  and  produces  the  training 
materials  to  be  used  in  the  training  system.  Delivery  takes  as  input  training  system  media 
allocations  from  the  design  process  and  training  materials  from  the  production  process  and 
as  output  provides  training  to  the  students.  Evaluation  provides  feedback  for  the  training 
system.  It  takes  as  input  the  performance  of  training  systems  and  provides  as  output 
information  on  the  adequacy  of  analysis  and  design  decisions  and  of  production  and 
delivery  practices. 
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2 .  Main  Training  Directives  and  Instruction 

Most  of  the  current  DoD  training  and  education  policy  is  described  in  two  DoD 
directives  and  one  DoD  instruction:  DoD  Directive  1322.18,  "Military  Training,” 
9  January  1987  (Reference  [4]);  DoD  Directive  1430.13,  "Training  Simulators  and 
Devices,"  22  August  1986  (Reference  [5]);  and  DoD  Instruction  1322.20,  "Development 
and  Management  of  Interactive  Courseware  for  Military  Training,"  14  March  1991 
(Reference  [6]).  Other  directives  and  instructions  apply,  but  these  three  documcne:  provide 
the  foundation  of  current  policy  and  arc  generally  considered  when  DoD  training  and 
education  policy  is  being  discussed. 

Summaries  of  policies  and  procedures  from  these  directives  and  instruction  are 
contained  in  tlw  next  three  subsections.  A  discussion  of  the  directive  on  military  training  is 
included  because  it  is  central  to  the  understanding  of  training  options.  The  training 
simulators  and  devices  directive  is  discussed  because  ICW  sometimes  includes  or  is 
included  in  such  simulators  or  devices,  and  often  simulates  actual  equipment  in  training, 
e.g.,  maintenance  training.  The  ICW  instruction  is  discussed  because  it  is  an  important 
management  initiative  to  lower  the  cost  of  maintaining  and  disseminating  ICW  technology. 

a.  DoD  Directive  1322.18,  '^Military  Training'* 

This  directive  establishes  DoD  policy  for  training  military  personnel  and  military 
units.  It  has  only  one  policy  statement  which  states  that  "it  is  DoD  policy  to  provide 
military  training  programs  for  the  total  force  that  support  force  readiness  and  use  resources 
efficiently."  Particularly  in  individual  training,  the  Directive  requires  that  the  the  most 
cost-effective  type  of  training  (institutional  or  on  the  job)  and  method  of  instruction  be 
used.  Also,  it  stipulates  that  "instructional  methods  such  as  individualized  instruction  that 
minimize  time  to  meet  course  objectives  shall  be  considered  for  use." 

Of  particular  interest  to  this  examination  of  ICW  technology  as  an  instructional 
delivery  approach  are  the  following  guidelines: 

•  "Vigorous  research  programs  shall  be  conducted  to  develop  innovative  training 
technologies  that  make  military  training  more  effective  and  efficient" 

•  "Simulators  and  training  devices  shall  be  used  when  they  effectively  and 
economically  supplement  training  on  actual  equipment.  .  ."  with  improved 
readiness  the  primary  criterion  for  their  use,  and  with  O&S  savings  an 
important  secondary  consideration. 

•  "Computo’-based  instructiem  shall  be  considered  when  analysis  determines  that 
its  use  would  be  effective  and  efficient" 
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•  "The  use  of  training  technology  shall  be  emphasized  in  solving  the  unique 
training  problems  of  tJw  Reserve  components." 

b.  DoD  Directive  1430.13,  "Training  Simulators  and  Devices" 

This  directive  establishes  policy  for  simulator  and  training  device  development, 
acquisition,  and  use,  it  authorizes  the  use  of  simulators  and  training  devices,  it  supports 
requirements  for  coincident  and  concurrent  development  of  training  systems  and  the 
military  system(s)  they  support,  and  it  emphasizes  the  integration  of  simulators  and  training 
devices  with  overall  training  systems. 

The  directive  states  that  policy  for  simulators  and  training  devices  grows  out  of  the 
general  policy  of  optimizing  the  operational  readiness  of  the  total  force.  Policy  for 
simulators  and  training  devices  is  articulated  in  four  categories  of  guidelines;  General, 
Development  Planning,  Acquisition,  and  Training  Effectiveness  Evaluation. 

The  directive  states  that  development  and  acquisition  of  simulators  and  training 
devices  within  the  military  services  shall  be  based  on  each  service's  training  requirements 
analysis  process,  and  shall  include  a  cost-benefit  analysis  of  alternatives  and  their  potential 
effects  on  Active,  Reserve  component,  and  inter-service  training. 

Under  acquisition  guidelines,  the  directive  states  that  alternatives  to  simulators  and 
training  devices  as  well  as  alternative  simulators  and  training  devices  shall  be  evaluated  by 
the  service  concerned.  This  evaluation  should  include  life-cycle  costs  and  benefits,  and 
flexibility  of  the  system  in  response  to  changes  in  the  system  to  be  maintained  and  in 
training  location,  method,  or  load.  Finally,  it  mandates  that  commercial  practices, 
equipment,  and  software  may  be  used  when  military-specific  requirements  do  not  exist 

c.  DoD  Instruction  1322.20,  "Development  and  Management  of 
Interactive  Courseware  for  Military  Training" 

The  main  goal  addressed  by  this  instruction  is  more  cost-effective  use  of  ICW  for 
military  training.  The  instruction  concentrates  on  meeting  this  goal  by  designing  and 
mandating  the  implementation  of  procedures  that  enhance  the  efficient  distribution  of  ICW 
materials  and  information.  These  procedures  include  requiring  certain  standards  for 
developing:  ICW  materials,  an  automated  information  system  for  archiving  ICW  materials, 
and  archiving  procedures  (such  as  keeping  copies  of  the  course-authoring  software)  that 
will  reduce  the  expense  of  modifying  and  transferring  this  technology  throughout  DoD.  In 
summary,  this  instruction  implements  procedures  that  lower  the  effort  (cost)  of  both 
finding  out  about  already-developed  ICW  and  transferring  ICW  to  other  organizations. 
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3.  Do  Present  Policies  Further  Systems  Activities  with  Respect  to  ICVV? 

The  main  hypothesis  to  be  examined  in  this  paper  is  that  ICW  technology  offers  a 
cost-effective  way  to  deliver  instruction  and  that  it  should  be  more  widely  used.  A  key 
question  is:  How  do  training  activities,  directives,  and  instructions  address  ICW 
technology  in  training  delivery?  Since  the  choice  of  instructional  medium  is  made  in  the 
design  activity,  we  will  evaluate  how  design  is  addressed  in  general,  and  how  technology 
such  as  ICW  is  addressed  in  particular. 

Design  is  addressed  by  all  three  documents  previously  mentioned.  The  directive 
on  simulators  and  training  devices  states  that  alternatives  to  simulators  and  training  devices 
as  well  as  alternative  simulators  and  training  devices  shall  be  evaluated.  It  goes  on  to  list 
four  specific  elements  that  are  to  be  included  in  each  evaluation.  The  directive  on  mUiiary 
training  states  that  systematic  procedures  shall  be  used  to  design  individual  training.  It 
goes  on  to  list  seven  activities  that  shall  be  included  in  these  procedures,  including 
determining  the  proper  allocation  of  training  to  institutional  training  and  on-the-job  training 
and  choosing  the  most  cost-effective  methods  of  instruction.  The  instruction  on  ICW 
attempts  to  ensure  that  usability  be  seriously  considered  at  the  design  stage  by  stating  that 
ICW  shall  be  designed  with  an  open  architecture  and  that  it  be  developed  to  promote  its 
cost-effective  use,  its  portability  among  DoD  components,  and  its  efficient  distribution  to 
all  potential  users. 

Two  of  the  documents  support  innovation  and  progress  in  the  design  of  training 
programs.  The  military  training  directive  states  that  vigorous  research  programs  shall  be 
conducted  to  develop  innovative  training  technologies  that  make  military  training  more 
effective  and  efficient  It  also  states  that  the  use  of  training  technology  shall  be  emphasized 
in  solving  the  unique  training  problems  of  the  Reserve  components.  Finally,  it  states  that 
computer-based  instruction  shall  be  considered  when  analysis  deiennines  that  its  use  would 
be  effective  and  efficient.  The  ICW  instruction  is  strongly  motivated  by  the  need  to 
promote  new,  more  effective,  and  more  efficient  approaches  to  training.  It  should 
accelerate  the  use  of  ICW  by  promoting  portability,  freedom  from  royalty,  license,  and 
other  run-time  fees,  and  the  preparation  and  storage  of  master  copies  for  easy  reproduction 
and  distribution  of  ICW  materials. 

In  summary,  these  documents  address  instructional  design,  and  specifically 
encourage  ICW  where  appropriate.  Consistent  wdth  this  encouragement,  DoD  has  been  a 
leader  in  the  develt^ment  of  instructional  technology  as  reported  in  reviews  by  Fletcher  and 
Rockway  [7]  and  the  Office  of  Technology  Assessment  [8].  Areas  where  DoD  has 
pioneered  and  provided  technology  leadership  include:  computer-assisted  instruction. 
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computer-managed  insiruciion,  intelligent  instructional  systems,  interactive  videodisc 
instruction,  computer-assisted  testing,  training  device  development,  applications  of  high- 
fidelity  simulators  such  as  flight  trainers,  applications  of  part-task,  lower-fidelity 
simulations  such  as  maintenance  trainers,  courseware  portability,  simulation  networking, 
and  systems  approaches  to  instructional  systems  development.  Other  agencies  of  the 
government  and  the  civilian  sectors  have  adopted  these  DoD  innovations  for  their  own  use. 

Given  that  ICW  technology  is  encouraged  by  these  DoD  policies,  let's  look  at  how 
widely  these  tools  are  used  by  DoD. 

C.  CURRENT  TRAINING  METHODS 

We  were  able  to  find,  collect,  and  analyze  Navy  data  provided  from  Chief,  Naval 
Education  and  Training  (CNET)  databases  on  the  current  use  of  self-paced  ICW.  On 
balance,  we  believe  the  evidence  obtained  shows  significantly  less  use  of  ICW,  and 
particularly  self-paced  ICW,  than  is  desirable  given  the  cost  savings  reported  in  the  next 
section.  Conclusive  evidence  on  the  current  use  of  self-paced  ICW  for  the  other  military 
departments  could  not  be  obtained  within  the  constraints  of  this  project  Our  discussions 
with  organizations  within  the  Army  and  Air  Force  that  collect  training  data  indicated  that 
their  data  were  not  available  in  a  machine-readable  form  that  could  be  easily  analyzed.^ 

The  Navy  data  is  presented  below  in  Tables  4  through  7.  We  combined  1989  data 
on  Navy  courses  with  1985  data  from  a  prior  study  (Byrnes  and  Schoeck  (93)  to  examine 
some  trends  over  time.  Both  1989  and  1985  data  were  extracted  from  the  Navy  Integrated 
Training  Resource  Analysis  System  (Nl  l  KAS).  Additional,  more  detailed  analyses  of  the 
1989  data  are  provided  in  Appendix  A. 

Table  4  shows,  for  each  type  of  course,  the  percentage  of  courses  that  have  each  of 
the  different  methods  of  instruction  shown.  Table  5  shows,  for  each  type  of  course,  the 
proportion  of  student  load  or  Average-on-Board  (AOB)  for  each  method  of  instruction. 

The  methods  of  instruction  highlight  two  aspects  of  this  instruction;  whether  or  not 
it  was  in  some  way  computer-mediated  and  whether  the  instruction  was  individualized 
through  self-pacing  and  self-sequencing.  Computer  mediation  may  involve  only 
management  of  the  course  (tracking  grades,  allowing  exams  to  be  taken  on  demand),  or  the 
ctnnputer  may  be  a  major  vehicle  for  delivering  instruction. 


^  DoD  Instniction  1322.20  (Reference  [6])  nuuidates  the  develc^ment  of  the  Defense  Instructional 
Technology  Information  System,  which  should  make  this  infwmation  more  easily  avai'aUe  in  a 
consistent  format. 
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T«bi«  4.  Courss  Counts  and  Parcantagas  by  Mathod  of  Instruction. 

FY  1989 


Percentage  by  Method  Instruction* 


Type  Course 

Self-Paced, 

Instructor- 

Managed 

Self-paced, 

Computer- 

Managed 

Group-Paced, 

Computer- 

Assisted 

Group-Paced 

(Lock-Step). 

Instructor 

Managed 

Missing/Oiher 

Total 

Courses 

Initial 

2.2 

4.1 

25.5 

68.2 

0.0 

314 

Advanced 

0.7 

0.2 

1.6 

97.3 

0.3 

2.768 

Tcam/Fleet 

1.2 

0.0 

0.4 

98.1 

0.3 

1,426 

Total 

0.9 

0.4 

2.9 

95.5 

0.3 

4.508 

*Perceiitage(  add  to  100  acrou  the  rows. 


Tabla  5.  Studant  Loads  (AOB)  and  Parcantagas  by  Mathod  of  Instruction, 

FY  1989 


Percentage  by  Method  erf  Instrocticin* 


Gioup-f^oed 


Type  (bourse 

Seif-Paced, 

Instnictcr- 

Managed 

Self-paced, 

Cemputer- 

Managed 

Group-Bkxd, 

Ccxnputer- 

Assisted 

(Lock-Step), 

Instructor- 

Managed 

Missing/Otber 

Total  Load 

Initial 

I.l 

3.4 

44.7 

50.7 

0.0 

23.048 

Advanced 

0.9 

0.2 

4.3 

94.2 

0.4 

16,002 

T:am/Fleet 

0.7 

0.0 

0.4 

98.4 

0.5 

4,070 

Total 

1.0 

1.9 

25.5 

71.4 

0.2 

43.119 

*PercenUgef  add  to  100  acrou  tbe  rowi. 


The  interesting  data  are  in  initial  skill  training  courses,  since  that  is  where  most  of 
the  large,  high-throughput  courses  are  located.  In  these  data,  32%  of  the  courses  and  49% 
of  the  AOB  are  in  self-paced  or  computer-mediated  instruction,  or  both.  That  is,  these  ate 
not  lock-step,  instructor-managed  courses.  If  these  data  can  be  interpreted  in  this 
straightforward  manner,  and  if  they  are  representative  of  the  Air  Force  and  Army,  then  it 
would  seem  there  is  significant  penetration  of  ICW  technology,  and  that  the  level  of 
training  expenditures  is  lower  than  it  would  otherwise  be,  though  not  as  low  as  possible. 

However,  such  straightforward  interpretation  is  difficult  for  the  group-paced, 
computer-assisted  instruction  (CAI)  category.  The  problem  of  interpreting  this  category  is 
that  the  easiest  cost  savings  to  identify  in  ICW  are  reductions  in  student  time,  which  occur 
through  instructional  individualization  in  general  and  self-pacing  in  particular.  These 
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savings  are  less  likely  to  be  realized  in  a  group-paced  environment.'*  That  is,  if  self-pacing 
is  the  characteristic  of  these  training  courses  that  generates  cost  avoidance,  then  only  6%  of 
tte  initial  training  courses  and  5%  of  the  initial  training  load  benefited  from  cost  avoidance 
gei^rated  by  ICW. 

In  Tables  6  and  7,  an  attempt  is  made  to  compare  1985  and  1989  data  to  see  if  there 
are  discemable  trends.  Unfortunately,  the  1985  course  counts  are  not  presented  in  as 
disaggregated  a  form  as  the  1989  data,  so  in  Tables  6  and  7  the  1989  data  are  aggregated 
accordingly.  Also,  the  1985  data  use  maximum  course  input  as  a  size  estimator,  so  the 
load  comparison  is  not  clear. 


Table  6.  Comparison  of  1980  and  1985  Course  Counts 


Year 

Tvoc  Course 

Percentage  by  Method  of  Instruction 

Total 

Courses 

Self-P^ 

HH 

1989 

Initial 

6.4 

93.6 

0.0 

314 

Advanced 

0.8 

98.9 

0.3 

2,768 

Team/Fleet 

1.2 

98.5 

0.3 

1,426 

Total 

1.3 

198.4 

0.3 

4,508 

1985 

Initial 

32.5 

67.5 

NA 

412 

AdvaDoed 

1.2 

98.8 

NA 

4,200 

Team/Fleet 

2.2 

97.8 

NA 

1,933 

Total 

3.4 

96.0 

0.6 

6,585 

Notes;  NA  means  not  available.  Lock-step  data  for  1989  are  an  auregation  of  tbe 
groop-paoed,  computer-assisted  and  tbe  gronp-paced.  instmetor-managed  columns 
in  Table  4. 


A  comparison  of  the  1989  and  1985  data  in  Table  6  is  interesting.  One  key  item  is 
that,  in  1985, 33%  of  the  advanced  school  courses  were  self-paced,  while  in  1989  only  6% 
of  those  courses  were  self-paced.  Since  self-pacing  for  individual  courses  results  in  less 
time  under  instruction,  it  is  not  clear  why  this  change  has  occurred.  It  was  most  probably 
the  result  of  a  CNET  policy  decision  in  the  mid-1980s  against  use  of  self-pacing,  resulting 
in  efforts  to  convert  self-pacred  courses  back  to  group-pacing.^ 


*  A  similar  obiervatioD  is  made  in  Orlansky  [10],  p.  400402. 

^  See  Sectioo  rV.B.2  of  this  paper  for  possible  reasons  for  this  policy. 
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TabI*  7.  Comparison  of  1989  Studant  Load 
with  1985  Avaraga  Maximum  Yaariy  input  Capacity 


PercentaKC  by  Method  of  Instruction 

Year 

Type  Course 

Self-P*3ed 

Lock-Step  MtssinR/fXber 

Total 

Courses 

1989 

(Student  Load) 

Initial 

4.6 

95.4 

0.0 

23.047 

Advanoed 

1.1 

98.5 

0.4 

16.002 

Team/Fleet 

0.7 

98.8 

0.5 

4.069 

Total 

2.9 

96.9 

0.2 

43.120 

1985 

(Average  Maximum 
Yearly  Input 
Capacity) 

Initial 

69.1 

30.9 

NA 

1,111 

Advanced 

2.9 

97.1 

NA 

86 

Team/Fleet 

3.4 

96.6 

NA 

388 

Total 

22.4 

77.6 

NA 

239 

Notes:  NA  meant  not  available.  Lock-step  data  for  1989  are  an  afgrc(at»n  of  Ibe  {roup-paced, 
computer  assisted  and  the  {roup-paced,  iostnKior-maoa{ed  columns  in  Table  S. 


In  summary,  the  available  data  indicate  that  the  advantages  of  self-pacing  in 
lowering  student  instructional  time  have  not  been  embraced  by  the  Navy.  Unfortunately, 
definitive  quantitative  findings  and  conclusions  with  respect  to  DoD-wide  ICW  usage  are 
difficult  to  generate  due  to  lack  of  data  for  tte  Army  and  Air  Force.  However,  qualitative 
assessments  by  Array  and  Air  Force  training  experts  do  not  suggest  extensive  use  of  ICW 
and  self-pacing  in  these  services.  As  can  be  seen  from  the  Navy  data,  the  use  of  self¬ 
pacing  in  initial  training  is  not  as  great  as  might  be  expected  and  it  seems  to  have  decreased 
during  1985-1989. 
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in.  COST  SAVINGS  FROM  ICW 


This  section  provides  first-order  estimates  of  the  cost  savings  that  may  be  available 
through  the  insertion  of  ICW  technology  in  place  of  existing  lock-step  technology.  Based 
on  a  set  of  what  we  believe  are  conservative  assumptions,  this  analysis  suggests  that  more 
extensive  application  of  ICW  technology  to  individual  training  wiu  result  in  both  financial 
and  personnel  savings.  The  analysis  estimates  annual  savings  of  $130  million  to  $160 
million  if  ICW  were  applied  to  specialized-skiU  training  only.  These  savings  result 
primarily  from  less  student  time  spend  under  instruction,  hence  a  lower  training  load  to 
achieve  a  given  output  of  trained  personnel.  Training  load  would  decrease  by  9-10 
thousand  personnel. 

The  analysis  that  supports  these  estimates  includes  a  review  of  prior  research 
literature  on  the  costs  and  benefits  of  different  training  technologies,  specifically  research 
on  the  percentage  cost  and  time  savings  from  ICW.  Savings  estimates  are  developed  in 
two  stages.  The  first  stage  applies  these  savings  percentages  to  easily  identifiable,  direct 
costs  of  specialized-skiU  training  for  active  forces.  The  second  stage  involves  estimating 
savings  to  cost  of  training  Reserve  components,  as  well  as  less  direct  training  costs  such  as 
Base  Operating  Support  A  more  detailed  analysis  of  Navy  Initial  Skill  data  is  presented  in 
Appendix  A 

Each  stage  presents  a  range  of  estimates  based  on  different  assumptions  about  the 
range  of  time  savings  and  the  proportion  of  courses  to  which  these  savings  apply.  In 
addition,  several  other  trends  are  discussed  that  have  implications  for  the  size  of  these 
savings  estimates. 

A .  COST  AND  TIME  SAVINGS  ESTIMATES  FROM  THE  RESEARCH 
LITERATURE 

The  potential  of  some  form  of  ICW  as  an  instructional  medium  has  long  been 
recognized.  However,  there  have  also  been  questions  about  whether  this  potential  could  be 
realized  in  a  cost-effective  manner.  This  uncertainty  has  led  to  a  wide  variety  of  studies  as 
new  variants  of  this  technology  (e.g.,  computer-based,  computer-aided,  or  computer- 
managed  instruction,  interactive  videodisc  instruction,  and  multimedia  instruction)  have 
been  introduced.  This  section  briefly  describes  several  citations  in  the  literature,  and 
extracts  cost  and  time  savings  estimates  for  use  in  our  rough  cost  estimate. 
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1 .  Literature  Reviews 


The  three  main  sources  used  here  are  Orlansky  [10],  Fletcher  [11],  and 
DeBIoois  [12].  All  these  documents  are  themselves  summaries  of  previous  cost, 
effectiveness,  and  cost-effectiveness  studies  of  ICW.  For  example,  Fletcher  summaries  28 
studies  (13  military  training,  4  industrial  training,  and  11  higher  education  studies)  and 
DeBIoois  summarizes  30  studies  (6  military).  Orlansky  summarizes  several  studies  that  are 
themselves  summaries  of  other  studies.  There  is  some  overlap  in  the  studies  summarized 
by  DeBIoois  and  Fletcher;  9  are  common  to  both  reports. 

2.  Cost  Drivers,  Costs,  and  Savings 

For  the  cost  analysis  undertaken  here,  we  need  estimates  of  both  the  benefits 
(savings)  of  ICW  and  the  investment  necessary  to  obtain  those  savings. 

A  major  cost  driver  for  training  is  student  pay  and  allowances;  hence,  an  important 
potential  source  of  savings  is  the  reduced  time  students  require  to  pass  through  ICW 
courses.  A  major  reason  for  faster  course  completion  is  that  ICW  courses  are 
individualized,  and  the  pace,  content,  and  sequence  of  instruction  is  tailored  to  the  needs 
and  capabilities  of  each  student.  Because  of  this  individualization,  as  opposed  to  group¬ 
pacing  or  lock-stepping  students  through  courses,  students  complete  courses  sooner  and 
student  pay  and  allowances  attributable  to  training  are  less  than  they  would  otherwise  be. 
Also,  the  average  number  of  students  in  training  at  any  given  time  can  be  reduced. 

With  respect  to  cost  avoidance  through  decreased  student  instructional  time,  the 
cited  works  show  substantial  consistency.  Orlansky  ([10],  p.  400)  reports  a  median  30% 
savings  in  student  time,  Fletcher  ([11],  p.  III-23)  reports  a  31%  savings  in  student  time, 
and  information  from  DeBIoois  [12]  can  be  used  to  estimate  a  similar  savings  in  student 
time  (26%  mean,  30%  median;  removing  the  studies  duplicated  in  Fletcher  does  not  change 
this  result  jq)preciably).  The  figure  of  30%  is  used  as  the  starting  point  for  our  analysis. 

The  expense  that  must  be  incurred  to  produce  these  savings  is  not  as  well 
summarized  in  the  cited  reports.  However,  by  examining  several  of  the  primary  sources,  a 
case  can  be  made  that  the  life-cycle  costs  of  ICW  are  no  greater  than  the  costs  of  the 
technologies  they  replace.*  For  example,  Fletcher  enumerates  seven  studies  that  report 
comparisons  of  initial  investment,  operating  and  support,  and/or  full  life-cycle  costs 


*  Some  cate  must  be  taken  with  tbis  result,  because  smne  of  the  original  instructional  delivery  methods 
involved  actual  equipment  whose  costs  are  usually  high,  and  some  involved  lecture-like  delivery 
qqiroaches  where  course  development  costs  are  tower. 
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betwef  n  then-existing  training  and  ICW  training  and  that  do  not  include  cost  savings  due  to 
less  student  time.  In  each  case,  the  comparison  favored  ICW.  He  also  reports  two  studies 
where  cost  comparisons  include  cost  savings  due  to  less  student  time.  These  also  favor 
ICW,  and  continue  to  do  so  even  when  student  cost  savings  are  removed. 

3 .  Summary 

In  summary,  these  studies  suggest  that  greater  use  of  ICW  should  result  in  lower 
costs  as  well  as  fewer  personnel  in  training  at  any  given  time.  The  net  life-cycle  costs  of 
the  ICW  technologies  themselves  do  not  appear  to  be  greater  than  existing  technologies.^ 
Therefore,  when  ICW  technologies  are  applicable  and  used  in  ways  that  allow  the 
individualization  inherent  in  these  technologies  to  be  expressed  in  different  course 
completion  times,  the  30%  student  time  savings  (hence  savings  in  student,  instructor,  and 
support  pay  and  allowances)  is  not  offset  by  higher  ICW  costs. 

The  next  question  is:  Given  the  student  time  savings  and  other  benefits  found  in  the 
literature,  if  ICW,  and  particularly  self-paced  ICW,  achieved  wider  use,  what  sort  of 
potential  savings  might  be  available? 

B.  POTENTIAL  SAVINGS 

In  order  to  estimate  overall  cost  savings  from  the  introduction  of  ICW  technology 
into  individual  training,  we  need  additional  information  beyond  that  discussed  so  far.  In 
particular,  we  need  to  know  what  percentage  of  the  individual  training  budget  is  subject  to 
savings  from  the  introduction  of  ICW. 

We  developed  an  estimate  of  the  potential  savings  from  ICW  introduction  in  two 
stages.  First,  we  considered  the  direct  costs  of  specialized  skill  training  for  active  forces. 
Within  specialized  skill  training,  we  estimate  the  percentage  of  training  load  in  courses  that 
might  be  converted  to  ICW  and  the  percentage  of  student  cost  savings  that  might  result 
from  such  conversions.  Wide  ranges  of  course  conversions  and  student  cost  savings  are 


It  could  be  argued  diat  life-cycle  cost  comparisons  are  less  i^ropriate  in  a  situation  where  one  training 
delivery  system  already  exists,  say,  small  group  instruction  on  actual  equipment,  and  the  alternative  is 
ICW  technology.  Because  parts  of  the  existing  approach  already  exist  (are  flxed),  the  correct 
comparison  would  then  be  the  variable  cost  of  the  existing  approach  versus  the  total  cos*  of  the 
alternative  apiMoach.  This  might  be  correct  in  special  cases,  but  as  has  been  aigued  elsewhere  (b'uUivan 
[13]),  it  is  quite  reasonable  to  believe  that  important  aspects  of  operating  procedures,  systems,  and 
platforms  are  changing  at  a  rate  rapid  enough  that  there  are  very  few  fixed  procedures  or  equipment  for 
more  than  a  few  years.  As  long  as  such  changes  are  occurring  rapidly,  both  alternatives  should  be 
measured  by  life-cycle  c(»ts. 
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used  because  the  savings  obtained  from  converting  a  single  course  may  be  lost  when 
sending  students  through  a  sequence  of  courses,  not  all  of  v^  hich  are  self-paced.  A  detailed 
explanation  of  how  those  savings  may  be  lost  is  given  later. 

Second,  we  included  cost  allocations  that  account  for  Reserve  Components,  Base 
Operating  Support  (BOS),  Temporary  Duty  (TDY),  Permanent  Change  of  Station  (PCS), 
and  Army  One  Station  Unit  Training  (OSUT).  These  estimates  are  described  separately 
because  of  their  lower  precision. 

Three  assumptions  are  maintained  through  this  analysis.  First,  we  assume  that  the 
extra  cost  of  ICW  is  balanced  by  reductions  in  instructor  pay  and  allowances  and 
acquisition  and  maintenance  expense  for  actual  equipment.  Thus,  these  values  arc  not 
included  in  the  savings  estimates. 

Second,  students  are  trained  to  given  criteria  regardless  of  the  instructional  delivery 
method  used.  That  is,  whether  ICW  or  lock-step  instruction  is  used,  students  learn  the 
same  skills  and  knowledge  and  are  therefore  equally  productive  in  their  assignments.  This 
assumption  will  be  further  discussed  in  Section  IV, 

Third,  the  standard  sequence  of  enlistment,  basic  training,  initial  skill  training,  and 
initial  assignment  will  be  maintained.  It  may  be  that  additional  training  savings  could  be 
obtained  if  more  entering  personnel  were  routed  from  basic  training  through  an  initial 
assignment,  and  then  back  for  initial  skill  training  (see  Wilson  and  Horowitz  [14]).  In  the 
latter  case,  it  might  be  that  ICW  could  be  used  to  augment  the  training  received  in  the  initial 
assignment  and  lower  the  training  costs.  However,  this  more  difficult  analysis — difficult 
because  readiness  is  also  being  traded  for  potentially  lower  costs — is  not  pursued  here. 

1.  The  Direct  Costs  of  Specialized-Skill  Training 

The  direct  cost  of  individual  training  estimated  for  fiscal  year  1990  is  $12,889 
million,  as  shown  in  Subtotal  I  in  Table  1.*  Of  that  amount,  $6,035  million  is  for 
specialized-skill  training,  of  which  $2,108  million  (35%)  is  student  pay  and  allowances.’ 
The  breakout  within  specialized  skill  training  is  shown  in  Table  8.  The  first  savings 
estimate  is  generated  from  student  pay  and  allowances. 


^  Some  portion  of  die  TDY  costs  are  direct,  but  we  did  not  to  try  to  determine  how  to  allocate  them. 

’  Estimates  of  the  proportion  of  direct  training  costs  that  are  student  pay  and  allowances  were  provided  to 
us  by  die  Training  Performance  and  Data  Center  (TPDQ. 
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Table  8.  Direct  Costs  of  Specialized-Skiil  Training 


Co.st  in  Millions  of  1989  Dollar.s 

Student  Pay  and 

Student  Pay  and 
Allowances  as  a 

Catesory 

Direct  Costs 

Allowances 

Percentage  of  I  ota! 

Army 

2.517.2 

859.3 

34 

Navy 

2.046.5 

811.3 

40 

Marine  Corps 

511.5 

220.0 

43 

Air  Force 

960.0 

217.8 

23 

Total 

6.035.2 

2.108.4 

35 

2 .  Savings  Estimates 

An  overly  optimistic  cost  avoidance  estimate  would  be  to  take  the  student  time 
savings  factor  of  30%  and  apply  it  to  the  entire  student  pay  and  allowances  figure  to 
generate  a  savings  estimate  of  $633  million.  However,  there  are  several  reasons  to  believe 
this  would  be  inappropriate.  For  example,  for  small  throughput  courses,  the  fixed  cost  of 
ICW  media  preparation  would  tend  to  overwhelm  the  time  savings.  Also,  there  is  probably 
some  irreducible  minimum  amount  of  waiting  time  during  training  that  prevents  the  30% 
savings  in  instruction  time  in  an  individual  course  from  translating  perfectly  to  a  30%  cost 
savings  for  instructional  sequence.  Finally,  we  are  not  sure  what  percentage  of  the  student 
load  is  in  courses  that  are  susceptible  to  cost  savings  through  conversions  to  ICW. 

The  different  savings  estimates  we  developed  depend  on  different  assumptions 
about  the  proportion  of  student  load  that  can  be  converted  to  ICW,'®  and  the  percentage  of 
student  costs  that  can  be  saved  through  use  of  ICW.  These  results  are  presented  in 
Table  9.  For  example,  if  the  student  time/cosi  savings  percentage  is  25%,  and  the 
proportion  of  student  load  that  can  be  converted  is  25%,  then  the  annual  cost  avoidance 
would  be  $132  million. 


'  ®  It  is  important  to  note  that  there  are  several  courses  or  course  sequences  that  contain  disproportionate 
amounts  of  the  specialized-skill  training  load.  For  example,  2%  of  the  Army  uaining  load  is  in 
Administrative  Specialist,  1%  of  the  Air  Force  load  is  in  Apprentice  Communications  Computer 
Systems  Operator,  and  4%  of  the  Navy  training  load  is  in  Avionics  Technician  A  school  (though  the 
latter  is  a  multi-threaded  coarse  due  to  training  for  different  airframes).  Convmion  of  a  small  number 
of  courses  on  sequences  such  as  these  would  bring  large  changes  in  student  time.  See  Reference  [1], 
p.  V-7  for  the  d^  from  which  these  figures  were  generated.  See  Appendix  A  for  a  more  detail^ 
breakdown  of  Navy  courses  and  training  loads. 
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Tabb  9.  Savings  In  Studant  Pay  and  AHoaranca  for  Spaclalixad-Sklll  Training 


Pwcentage  of  Student  Savings  cm  Millions  of  IXitlarsi  b> 

Time/Cosi  Saved  Hrrvcncage  of  Student  I.aad  Affcctol 


5% 

10% 

15% 

20% 

25% 

35% 

5% 

5 

11 

16 

21 

26 

32 

.3? 

42 

10% 

11 

21 

32 

42 

53 

6.3 

74 

15% 

16 

32 

47 

63 

79 

‘>5 

HI 

1  'T‘7 

20% 

21 

42 

63 

»4 

105 

127 

148 

25% 

26 

53 

79 

105 

132 

158 

184 

211 

30% 

32 

63 

95 

127 

158 

190 

1 

253 

35% 

37 

74 

ill 

!4g 

m 

22 1 

258 

295 

40% 

42 

S4 

127 

16«1 

211 

253 

295 

337 

In  this  example,  ihc  overall  student  lime  savings  would  be  6,25^  (0.25  x  0.25  = 
0.0625).  From  Table  2,  the  average  student  ’oad  is  18.1.8  thousand,  hence  the  personnel 
saving  would  be  1 1.5  thousand  (183.8  thousand  x  0.0625).” 

3.  Other  Factors  that  Affect  Savings 

Other  factors  may  influence  the  sixe  of  these  savings.  These  factors  affect  the 
fraction  of  cost  saved,  the  proportion  of  load  in  converted  courses,  and  the  amount  of 
savings. 

As  already  noted,  there  is  some  uncertainty  that  any  given  pcrccnugc  of  student 
lime  savings  translates  into  an  equivalent  percentage  of  cost  savings  in  student  pay  and 
allowances.  Also,  while  individual  course  components  can  be  shortened  on  average,  full 
course  sequences  that  are  mixtures  of  lock-step  and  self-paced  components  mean  that 
individual  component  time  savings  may  not  translate  into  course  sequence  time  savings. 
Both  these  factors  tend  to  move  the  most  likely  estiniate  toward  the  upper  left-hand  comer 
of  the  Table  9  matrix,  and  hence  towards  smaller  savings. 

However,  several  other  factors  suggest  that  the  savings  in  any  given  cell  in  the 
matrix  are  underestimates  of  to'al  savings.  These  factors  are  summarized  in  Table  10.  For 
example,  slightly  over  52%  of  the  Reserve  Component  training  load  is  in  specialized-skill 
training  ([1],  p.  6  and  V-2),  Therefore,  approximately  $485  million  of  the  Reserve 
component  pay  and  allowances  from  Table  1  could  be  added  to  the  student  pay  and 
allowances  from  which  these  savings  are  f^tored.  This  would  increase  savings  in  each 
cell  by  approximately  23%. 


*  *  Sec  Appendix  A  for  Navy  data  that  show  20-25%  of  total  student  load  is  in  a  small  number  of 
qpedalized-skiU  training  courses. 
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Other  factors  could  increase  the  amount  of  possible  savings.  For  example,  some 
component  of  BOS  would  vary  with  training  load;  approximately  50%  of  Army  OSUT 
training  is  spec'  lized-skill  training;  and  some  part  of  TDY  and  PCS  expense  is  dependent 
on  student  time.  While  cost  estimates  for  these  categories  are  less  precise,  rough  estimates 
and  their  justification  are  presented  in  Table  10. ‘2 


Tabis  10.  Additional  Savings  from  Othar,  indiract  Sourcas 


Savings  Source 

Percentage  Increase  in  Savings 

BOS* 

52% 

Reserve  Components  Pay  and  Allowances^ 

23% 

TDYandPCSc 

15% 

ArmvOSUTd 

4% 

*  Fifty  percent  of  S4,40S  million  in  FY  1989  dollars  BOS  in  Table  1  is  allocated  to 
specialized-skill  training  based  on  proportion  of  total  direct  costs.  Fifty  percent  of 
that  is  usumed  to  vary  with  student  time  (based  on  a  conservative  interpretation  of 
ongoing  IDA  research).  The  result  is  approximately  $1,101  million,  or  52%  of 
student  pay  and  allowances. 

^  As  explained  in  the  text,  $48S  million. 

^  Data  provided  by  TPE)C  report  $1,240.7  millioo  of  TDY  is  already  included  in  the 
$6,03S  million  in  specialized -skill  training.  If  25%  of  this  total  amount  varies 
with  student  time,  tto  yields  another  $310  million  in  savings. 

^  Student  pay  and  allowances  at  OSUT  are  $172.3  million,  ai^troximately  one-half  of 
OSUT  training  is  specialized  skill. 


The  level  of  precision  on  these  estimates  is  very  low.  However,  if  these  savings 
were  realized,  the  value  in  every  cell  in  the  Table  9  matrix  would  increase  by  a  total  of 
94%,  as  shown  in  Table  11.  Using  this  table,  if  the  actual  cost  savings  percentage  were 
20%  and  the  proportion  of  training  load  in  courses  that  can  be  converted  to  ICW  were 
20%,  then  the  annual  cost  avoidance  would  be  $164  million.^^ 

Personnel  savings  estimates  are  also  rough.  They  would  include  4%  (0.2  x  0.2  = 
0.04)  of  active  and  researve  load  (8.8  thousand),  and  an  unknown  number  of  BOS 
personnel. 


^  2  Oacc  again,  instructor  pay  and  allowances  are  assumed  to  be  part  of  the  onnparison  of  life-cycle  cost  of 
different  instructional  teclmology  options. 

*  ^  Tbe  range  of  the  e^imates  ($132-$164  million)  could  be  much  larger  by  reporting  the  student  pay  and 
allowance  costs  in  Table  9  at  the  20%  cost  and  load  levels  (S84  million);  and  the  total  costs  from  Table 
1 1  at  tbe  25%  cost  and  knd  levels  ($256  million).  However,  we  believe  the  extremes  in  this  range  are 
less  likely. 


Table  11.  Estimates  of  Total  Savings  in  Student  Pay  and  Allowance  for 

Specialized-Skill  Training 


Percentage  of  Student  Savings  (in  Millions  of  Dollars)  by 

Time/Cost  Saved  Percentage  of  Student  Load  Affected 


5% 

10% 

15% 

20% 

25% 

30% 

35% 

40% 

5% 

10 

20 

31 

41 

51 

61 

72 

82 

10% 

20 

41 

61 

82 

102 

123 

143 

164 

15% 

31 

61 

92 

123 

153 

184 

215 

245 

20% 

41 

82 

123 

164 

205 

245 

286 

327 

25% 

51 

102 

153 

205 

256 

307 

358 

409 

30% 

61 

123 

184 

245 

307 

368 

429 

491 

35% 

72 

143 

215 

286 

358 

429 

501 

573 

40% 

82 

164 

245 

327 

409 

491 

573 

654 

4.  Other  Trends 

Several  other  trends  indicate  that  even  the  estimates  in  Table  1 1  may  understate  the 
potential  savings  from  greater  conversion  to  ICW  technology.  First,  centralized  training 
may  become  more  difficult  because  future  Total  Force  plans  may  include  a  greater  reliance 
on  Reserve  components.  Second,  there  are  more  materiel  systems  to  be  maintained  and 
these  systems  are  technically  more  complex,  which  may  result  in  even  larger  training 
requirements;  hence,  more  savings  may  be  possible  through  the  use  of  ICW.  Third,  the 
cost  of  digital  electronic  technology,  an  underlying  ICW  cost  driver,  has  fallen  dramatically 
in  the  past,  and  will  continue  to  do  so  over  the  foreseeable  future.  These  factors  are 
discussed  in  the  following  subsections. 

a.  Future  Difficulties  in  Centralizing  Training 

A  potential  difficulty  in  centralizing  training  would  occur  if  there  is  an  increased 
reliance  on  Reserves  in  the  Total  Force.  For  example.  Desert  Storm  and  Desert  Shield  are 
offering  new  insights  about  the  relative  readiness  and  effectiveness  of  different  types  of 
Reserve  units.  If  a  greater  reliance  on  Reserve  components  becomes  policy,  it  could 
strongly  effect  the  choice  of  cost-effective  training  design. 

Reserve  components  have  a  limited  amount  of  time  to  train,  units  are  widely 
dispersed  throughout  the  country,  some  units  are  not  fully  equipped,  and  only  a  small  full¬ 
time  force  of  qualified  supervisors  and  trainers  exists  in  many  Reserve  units.  Relatively 


^  *  One  trend  that  might  lower  the  total  savings  is  overall  reduction  in  the  military  services.  That  is,  in  a 
steady  state,  a  20%  smaller  military  would  require  20%  less  training.  Howeva,  whatever  {^oportional 
savings  are  generated  are  still  available,  e.g.,  25%  time  savings  fw  whatever  the  student  load.  Also, 
decreasing  the  numbers  of  personnel  that  are  receiving,  giving,  or  supporting  training  while 
maintaining  training  output  means  that  a  givoi  personnel  ceiling  allows  a  larger  deployed  force. 
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more  Reserves  mean  that  it  may  become  more  cost-effective  to  deliver  standardized  training 
to  the  students  than  to  deliver  students  to  centralized  training  facilities.  The  ability  of 
individuals  to  receive  this  training  will  be  enhanced  if  the  training  has  the  flexibility  to  be 
delivered  at  any  time  and  in  any  location  the  student  is  available. 

It  has  been  argued  that  ICW  offers  great  potential  as  a  flexible,  decentralized 
instructional  delivery  method.  Walker  [15]  has  done  an  analysis  of  the  interactive 
videodisc  approach  to  decentralized  training  and  estimates  that,  over  a  three-year  life  cycle, 
interactive  videodisc  would  cost  35%  of  centralized  training  methods. 

If  this  analysis  is  borne  out,  it  has  further  implications  for  the  cost-effective  use  of 
ICW  in  centralized  individual  training.  If  Total  Force  policy  initiatives  that  imply  a  greater 
reliance  on  Reserve  components  are  implemented,  and  if  the  full  cost  of  developing  ICW 
cost  is  less  than  the  marginal  cost  of  centralized  training  for  these  personnel,  then  these 
technologies  will  exist  and  be  available  for  active  force  centralized  training  at  a  very  low 
cost.  This  makes  ICW  training  even  more  likely  to  be  a  cost-effective  alternative  to 
centralized  training. 

b.  The  Number  and  Complexity  of  Systems 

The  number  of  military  systems  has  been  increasing.  At  the  end  of  World  War  I, 
500  materiel  systems  were  fielded  by  the  the  U.S.  military.  At  the  end  of  World  War  n, 
this  number  had  increased  to  about  2,000,  and  currently  about  4,000  systems  are  either 
fielded  or  in  plaiming.  In  the  Array  there  are  about  0,78  large  systems  per  person — one 
wheeled  vehicle  for  every  four  people,  one  tracked  vehicle  for  every  20  people,  one  radio 
for  every  6  people,  one  generator  for  every  10  people,  and  so  on.  If  no  technological 
changes  were  made  in  the  complexity  of  military  systems,  their  quantity  and  variety  would 
by  themselves  substantially  increase  the  demands  on  military  training  to  provide  the  people 
needed  to  operate  and  maintain  them. 

The  technological  complexity  of  military  systems  is  increasing.  In  1939,  the 
volume  of  technical  documentation  required  for  the  J-F  Goose  "Catalina  Flying  Boat"  was 
525  pages  of  information.  In  1962,  the  volume  required  by  the  A-6A  Intruder  was  about 
150,000  pages  of  information.  The  volume  required  for  the  F-14  Tomcat  in  1975  was 
about  380,000  pages  of  information  and  estimates  for  the  B-1  bomber  in  1979  were  in  the 
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neighborhood  of  1,000,000  pages  of  information.  (All  technical  documentation  data  are 
from  Kline  and  Gunning,  [16].)  This  trend  will  doubtless  continue. 

The  demand  for  people  prepared  to  hold  jobs  classified  as  technical  or  highly 
technical  has  increased,  whether  due  to  the  number  or  complexity  of  weapons  systems. 
The  Army  projected  the  number  of  people  required  in  jobs  categorized  as  "Very  Technical" 
and  "Technical"  would  increase  by  about  6%  from  1984-1990,  and  the  Navy  projected  an 
increase  of  about  14%  in  "Highly  Technical"  and  "Technical"  categories  over  the  same 
period  (Binkin,  [17]). 

If  job  complexity  and  the  number  of  people  to  train  in  those  jobs  continue  to 
increase,  the  demands  on  training  resources  increase  concomitantly.  The  student  time 
savings  that  result  from  use  of  ICW  and,  especially  in  this  case,  the  considerable  savings 
that  result  from  the  substitution  of  ICW  simulations  of  devices  and  situations  for  actual 
equipment  and  situations  will  become  increasingly  important  (e.g.,  in  maintenance 
training). 

c.  Digital  Electronic  Technology  Costs 

The  hardware  and  software  costs  for  a  given  quality  and  quantity  of  ICW  will 
continue  to  fall  over  time  for  three  reasons.  First,  the  cost  of  computer  hardware  has  been 
falling  rapidly  over  time.  Second,  microcomputers  are  being  substituted  for  mainframes  as 
the  host  for  ICW,  and  these  microcomputers  are  cheaper  for  a  given  amount  of  computer 
power  than  are  mainframes.  And  third,  cheaper  hardware  means  that  hardware  can  be 
substituted  for  personnel  and  software  in  the  ICW  provision  process,  providing  a  lower 
cost  delivery  system,  or  allowing  greater  capability  at  the  same  price. 

Mainframe  Hardware  Costs.  A  wide  range  of  studies  has  demonstrated  that 
constant-dollar  prices  for  computer  hardware — central  processing  unit  (CPU)  and  random 
access  memory  (RAM) — for  a  given  amount  of  computing  power  on  mainframe  computers 
have  fallen  20-24%  per  year  since  1953.  (See  Triplett  [18]  for  a  survey  and  synthesis  of 
these  studies.)  Prices  for  disk  drives  have  fallen  almost  as  rapidly  (see  Cole  et  al.  [19]). 
This  results  in  an  order-of-magnitude  decrease  in  the  price  of  computer  systems  of  similar 
power  and  storage  every  9-10  years. 


^  ^  Whether  the  complexity  of  military  systems  necessarily  translates  into  increased  job  complexity  is  a 
topic  of  continuing  debate.  However,  several  suggestions  for  diminishing  the  effects  of  system  wd 
documentaticHi  cmnplexiiy  involve  greater  use  of  computer  technologies  for  documentation  storage; 
therefore,  future  maintenance  procedures  will  involve  using  the  same  equipment  normally  associated 
with  ICW. 
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Microcomputer  Hardware  Costs.  The  price  of  microprocessor  computer 
power  is  falling  at  the  rate  of  25-30%  per  year  (Bemdt  and  Griliches  [20]).  This  is  an 
order-of-magnitude  decrease  in  price-per-unit  power  every  7-8  years.  There  are  reasons  to 
believe  this  decline  will  continue  over  the  coming  decade  [21].  Also,  microcomputers  are 
significantly  less  expensive  per  unit  of  power  than  are  larger  computers.  Ein-Dor  [22] 
estimates  that  a  microcomputer  can  deliver  a  given  amount  of  power  at  a  price 
approximately  two  orders  of  magnitude  less  than  that  of  mainframes. 

Therefore,  as  a  given-price  microcomputer  becomes  more  powerful,  ICW 
applications  can  migrate  to  it  for  large  potential  cost  savings.  These  savings  have  already 
been  empirically  demonstrated  in  civilian  education  applications  (Fletcher,  Hawley,  and 
Piele  [23]). 

Hardware  Substitution.  Reductions  in  hardware  prices  may  appear  to  be  self- 
limiting  if  we  assume  that  current  combinations  of  hardware,  software,  and  courseware  are 
immutable.  For  example,  Kemner-Richardson  et  al.  [24],  p.  37,  cite  a  study  that  "reported 
in  their  cost  analysis  of  a  time  sharing  system  that  computer  equipment  represented  only 
about  28  percent  of  the  total  annualized  costs,  so  that  even  if  equipment  costs  declined  by 
one-third,  total  costs  would  be  decreased  by  less  than  10  percent”*^ 

The  point  missed  in  the  above  quote  is  that,  as  hardware  prices  fall,  cheaper 
hardware  will  allow  hardware  and  software  to  substitute  for  personnel,  to  produce  the  same 
or  superior  output  A  common  instance  of  this  son  of  behavior  is  buying  a  faster  computer 
in  order  to  complete  a  project — an  example  of  the  familiar  response  of  "throwing  hardware 
at  a  problem."  Another  example  would  be  using  sophisticated  course-authoring  software 


^  ^  Tbe  computer  power  advantage  of  miaocomputers  o«.  er  mainframes  is  not  as  great  as  the  two-order-of- 
magnitude  figure  would  indicate,  because  central  computers  can  be  shared  by  many  users,  and  each  of 
these  users  would  need  their  mainframe  terminal  replaced  by  a  microcomputer,  which  is  much  less 
shareable.  However,  tbe  combination  of  cost  issues,  such  as  faster  price  f^l  and  lower  cost  per  unit 
power,  and  amiability  issues,  such  as  greater  interactivity  with  tbe  user,  are  such  that  ICW  proviskn  is 
becoming  mote  and  more  microcmnputer-oriented. 

^  ^  See  Levin  and  Woo  [25],  or  Levin,  Glass,  and  Meister  [26]. 
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on  a  microcomputer  to  generate  ICW.i*  Quantitative  analysis  of  these  substitution 
possibilities  is  an  interesting  research  topic,  but  far  outside  the  scope  of  this  paper.*® 

It  should  be  emphasized  that,  as  with  the  mainframe  and  microcomputer  costs 
already  mentioned,  hardware  substitution  will  result  in  a  lower  price  to  produce  a  given 
level  of  training.  We  may  instead  choose  to  produce  more  expensive  training  systems  to 
deliver  more  effective  training — training  that  was  not  previously  provided  because  it  was 
not  perceived  as  cost-effective. 

Summary.  Declining  hardware  costs  and  substitution  of  hardware  for  personnel 
and  software  are  making  ICW  more  affordable.  These  changes  mean  IC!W  applications  are 
becoming  more  cost-effective.  Existing  ICW  applications  are  even  cheaper,  analyses  that 
concluded  ICW  was  not  quite  cost-effective  as  other  formal  teaching  methods  will  be 
reversed,  and  increasingly  demanding  requirements  may  be  more  affordable  met  with  ICW. 
This  has  profound  effects  on  the  design  of  training  systems  discussed  in  II.B.l. 


1 8  A  familiar,  non-training  example  of  a  more  ccnnplex  version  of  this  substitution  is  the  development  of 
grs^cal  user  interfaces  (GUIs),  first  cm  the  Xerox  Alto  and  Star  systems  (circa  1980),  then  on  the 
A{^)e  Lisa  and  Macintosh  (introduced  in  1983-84),  versions  of  Microsoft  Windows  (various  versions 
introduced  1985-1992)  for  IBM  PC  axtqiatibles,  and  in  X- Windows  and  Motif  for  Unix  platforms. 
More  powerful  CPUs,  more  memory,  and  better  displays  are  combined  with  extremely  complex 
software  (which  would  be  much  more  dillicull  to  develop  md  improve  without  inexpensive  hardware 
and  the  better  tools  deveicq)ed  because  of  the  existence  of  that  hardware)  to  make  using  cmnputers 
easier.  The  same  s(M  of  changes  are  occurring  in  ICW. 

*  ®  A  quantitative  measure  of  the  ease  with  which  one  input  (e.g.,  ccxnputing  power)  may  be  substituted 
for  other  inputs  in  the  production  process  is  called  the  elasticity  of  substitution.  It  measures  how 
changes  in  the  relative  prices  of  inputs  work  through  the  production  process  to  change  the  prppwtions 
of  inputs  used  to  produce  a  given  ouqrat.  This  relatkmship  is  constructed  to  lie  between  0  and  <»,  widi 
larger  values  indicating  easier  substitution.  One  result  available  from  the  literature  (Ferguson  [27], 
p.  104)  indicates  that,  as  long  as  the  elasticity  of  substitution  between  cmnputer  hardware  and  software 
in  the  production  of  ICW  is  greater  than  one  and  the  price  of  cmniHiting  power  ctmtinues  to  fall,  output 
growth  would  not  be  constrained  by  an  inability  to  substitute  h^virare  for  software.  However,  our 
willingness  to  purchase  this  output  may  be  constrained  by  its  total  cost,  and  it  is  not  clear  that  the 
elasticity  o(  substitutioo  is  in  fact  greater  than  one. 
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IV.  POLICY  ISSUES,  POLICY  OPTIONS, 
AND  RESEARCH  AREAS 


This  section  examines  two  issues.  The  first  issue  is  the  possibility  that  cost  and 
time  savings  for  individual  course  components  may  not  translate  into  savings  for  course 
sequences  or  as  part  of  the  total  process  of  training  and  assignment.  The  second  issue  is 
the  important  longer  term  effect  of  incorporating  significant  amounts  of  ICW  into 
training — the  fact  that  this  use  of  ICW  may  affect  the  requirements  for  the  amount  and 
kinds  of  knowledge  and  skills  that  the  military  wishes  to  deliver  to  students. 

A .  IMPEDIMENTS  TO  ICW  INCORPORATION 

Because  ICW  allows  self-pacing  and  other  forms  of  individualization,  the 
incorporation  of  ICW  technology  into  course  components  results  in  significant  savings  in 
student  time  as  better  students  assimilate  the  material  more  quickly.  Also,  the  studies 
previously  cited  indicate  that  both  initial  investment  and  operating  and  support  (O&S)  costs 
of  ICW  are  no  higher  than  the  technology  that  they  replace.  However,  while  self-paced 
ICW  technology  is  being  used  in  individual  training,  it  does  not  seem  to  be  used  as  widely 
as  these  potential  savings  estimates  would  indicate. 

Our  discussions  with  individuals  who  manage  and  analyze  training  yielded  several 
possible  explanations  for  this  phenomenoiL  The  two  most  common  explanations  dealt  with 
(1)  problems  in  combining  lock-step  and  self-paced  courses,  and  (2)  additional,  qualitative 
benefits  of  group-pacing  and  instructor-mediated  training.  We  do  not  believe  these  issues 
overwhelm  the  cost  savings  estimated  in  Section  m,  but  they  should  be  raised. 

1 .  Combining  Lock'Step  and  Self-Paced  Courses 

Although  course  components  can  be  shortened  through  ICW  while  still  training  to 
criteria,  full  courses  or  course  sequences  that  use  mixtures  of  lock-step  and  self-paced 
components  mean  that  individual  course  time  savings  may  not  translate  into  overall  time 
savings. 

A  hypothetical  example  would  be  the  following.  A  specialized  skill  training 
sequence  currently  has  three,  two- week,  lock-step  course  components  labelled  A,  B,  and 
C.  The  courses  follow  one  another,  so  they  start  two  weeks  apart.  If  course  B  is 
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converted  to  ICW,  the  number  of  days  it  will  take  individuals  to  obtain  the  knowledge 
disseminated  in  the  course  will  change  from  10  to  a  range  of  5-12.  Those  who  finish  early 
will  be  forced  to  wait  for  course  C  to  start,  while  those  who  take  longer  than  10  days  will 
miss  their  lock-step  class  and  have  to  wait  for  the  next  C  course  to  begin. 

A  similar  problem  exists  in  the  incorporation  of  training  in  the  larger  process  of 
personnel  movement  to  and  from  training.  For  example,  if  Navy  personnel  are  to  be 
assigned  to  a  ship,  but  cannot  repon  until  the  ship  is  in  port,  an  early  completion  of  their 
training  will  simply  mean  they  must  await  assignment  for  a  longer  period. 

If  either  or  both  of  these  situations  characterize  much  individual  training,  then  the 
cost-effectiveness  of  ICW  will  be  decreased.  However,  the  pervasiveness  of  these 
situations  needs  to  be  supported  or  refuted  with  evidence,  not  accepted  or  rejected  without 
examination. 

One  reason  to  think  that  this  problem  may  be  overstated  is  that  those  who  directly 
manage  training  do  not  necessarily  bear  the  full  cost  of  additional  student  time  under 
instruction.  This  is  because  training  managers  do  not  pay  students  from  their  budgets,  and 
higher  level  management  does  not  have  accepted,  incremental  cost-to-train  data  that  include 
student  pay  and  allowances. 

Because  many  training  managers'  budgets  do  not  include  student  pay  and 
allowances,  students  are  less  costly  to  them;  hence,  the  managers  will  not  necessarily 
optimize  the  amount  of  time  students  spend  in  training.  In  fact,  our  observation  has  been 
that  a  "small"  amount  of  free  time  by  students  yields  free  labor  for  necessary  tasks  in 
training  facilities.^^^  Also,  because  training  budgets  do  not  show  student  pay  and 
allowances,  it  is  harder  for  higher  level  management  to  observe  the  cost-effectiveness  of 
student  personnel  use. 

One  policy  initiative  that  is  attempting  to  better  address  this  issue  is  the 
DoD(Comptroller)  initiative  on  unit  cost  of  military  training.  "Unit  cost"  here  means  cost 
of  a  unit  of  output,  often  taken  to  mean  a  graduate  of  a  particular  set  of  courses.  This 
initiative  is  motivated  by  the  need  to  obtain  meaningful  cost-to-train  estimates  for  budgetary 
and  policy  purposes.  Such  cost  estimates  are  important  to  many  training  management 
decisions,  one  of  them  being  the  incorporation  of  ICW  technology  into  training. 


20  However,  when  the  free  or  waiting  time  f<x  students  becomes  "large"  or  "excessive,"  as  it  did  in  several 
Na^  training  facilities  in  the  eariy  1970s  and  early  1980s,  hitter  level  management’s  attention  results 
in  significant  inoentives  to  correct  this  imbalance. 
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2 .  Qualitative  Benefits  of  Group-Pacing  and  Instructor-Mediated  Training 

There  appear  to  be  three  main  qualitative  benefits  of  lock-step  and  instructor- 
mediated  training.  One  is  the  additional  military  training  and  discipline  realized  through  the 
processes  of  lock-step  training  with  military  instructors.  Unit  cohesion  may  be  greater  and 
discipline  more  evident  in  these  situations.  Second,  group-paced  training  may  result  in 
team-building  among  students  if  fast  learners  are  used  as  tutors  for  slow  learners.  Finally, 
instructor  mediation  often  results  in  additional  information  being  disseminated.  For 
example,  lessons  that  include  phrases  such  as  "Let  me  tell  you  how  it  is  really  done  ..."  or 
”We  deviate  from  these  procedures  when  .  .  may  provide  connections  between 
knowledge  and  skill  and  their  application  that  are  difficult  to  achieve  in  a  self-paced, 
computer-mediated  environment 

These  benefits  are  not  inherently  qualitative,  but  few  attempts  have  been  made  to 
measure  and  quantify  them.  They  are  widely  believed  and  reported  by  members  of  the 
military  training  communities,  and  for  this  reason  must  be  taken  seriously  as  credible 
benefits  fix>m  lock-step,  instructor-mediated  instruction. 

These  benefits  ate  not  incompatible  with  the  use  of  ICW  programs  (McCombs  and 
Dobrovolny  [28]).  However,  the  general  belief  that  these  benefits  are  available  only  from 
group-paced  instruction  may  continue  to  impede  the  adoption  of  ICW  in  militaiy  programs 
of  instruction. 

B .  HOW  DOES  ICW  AFFECT  THE  ACCUMULATION  OF  KNOWLEDGE? 

Two  interesting  questions  raised  by  the  possibility  of  significantly  more  ICW  being 
used  for  training  are:  Should  we  train  to  the  same  level  of  knowledge  as  with  other 
methods?  and  Should  we  train  aU  individuals  to  the  same  criteria,  or  encourage  some  to 
gain  more  knowledge  during  training? 

Individuals  in  training  accumulate  knowledge,  learn  to  apply  that  knowledge 
individually  to  perform  certain  tasks,  and  then  interact  collectively  to  perform  tasks  that 
contribute  to  readiness  or  warfighting  capability.  In  this  paper,  we  have  implicitly  assumed 
that  if  individual  training  provides  knowledge  that  results  in  passing  a  given  set  of  tests,  the 
performance  and  readiness  results  will  be  the  same  regardless  of  the  training  method. 
However,  even  if  this  assumption  is  true,  a  research  question  with  interesting  implications 
for  training  management  and  ICW  insertion  is  how  this  knowledge  accumulates  und^r 
various  instructional  i^)proaches,  and  at  what  cost 
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A  simple  visual  representation  of:  (a)  the  incremental  accumulation  of  knowledge 
and  (b)  the  costs  of  that  accumulation  is  presented  in  Figure  1 .  The  horizontal  axis  shows 
the  time  (t)  spent  by  one  individual  in  training.  The  vertical  axis  shows  both  the  value  of 
the  incremental  or  marginal  accumulation  of  knowledge  over  time,  labeled  MP  or  Marginal 
Product,  and  the  incremental  cost  of  the  time  spent  under  instruction,  labeled  MC  or 
Marginal  Cost 


Figura  1.  Individual  Knowledge  Accumulation  in  Lock>Step  Training 

The  marginal  cost  is  flat,  indicating  that  any  given  week  is  as  costly  as  the  last 
This  is  a  reasonable  assumption  since  the  main  course  costs  that  vary  with  student  input  are 
student  and  instructor  pay  and  allowances.  The  marginal  product  curve  is  based  on  the 
assumption  that  knowledge  accumulates  slowly  at  first  as  individuals  learn  what  seem  to  be 
a  series  of  unconnected  facts,  then  rapidly  as  the  knowledge  coalesces  into  a  basic 
understanding  of  the  subject  matter,  then  slowing  again  as  "nice  to  have"  bits  of  knowledge 
are  added.^* 

From  the  perspective  of  cost-effective  training,  students  should  be  trained  for  to 
time,  that  is,  until  the  MPq  curve  intersects  the  MCo  curve  as  shown.  As  long  as  the  value 
of  the  time  spent  in  class,  measured  by  MPq.  is  greater  than  the  cost  of  that  time,  measured 


^  ^  The  accumulation  of  the  ability  to  perform  activities,  such  as  actually  maintaining  a  weapon  system. 
may  have  a  diffeient  MP  curve.  However,  this  Jifference  between  knowledge  and  performance  occurs 
regardless  of  training  method,  and  we  have  no  a  priori  reason  to  believe  it  depends  chi  differences 
between  conventional  and  ICW  training  methods. 
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by  MCq,  training  is  cost-effective  and  should  be  undertaken.  When  the  value  is  less  than 
the  cost,  further  training  should  be  terminated. 

One  of  the  effects  of  lock-step  training  is  to  force  all  individuals  into  a  knowledge 
accumulation  pattern  similar  to  that  described  by  MPq  in  Figure  1.22  xhe  introduction  of  a 
self-paced  approach  such  as  ICW  might  result  in  patterns  such  as  those  shown  in  Figure  2. 
(Each  type  of  curve  has  the  same  interpretation  as  those  in  Figure  1 ,  though  they  do  not 
have  to  be  the  same  shape.)  Each  individual  accumulates  knowledge  at  a  different  rate.  If 
testing  to  criterion  is  used  to  pass  students  on,  the  areas  under  these  curves  would  be  equal, 
as  shown  by  the  integral. 


Time  (t) 
t. 

I 

J  MP  =  requirement  for  all  i 

0  I 

Figurs  2.  Individual  Knowledge  Accumulation  in  ICW  Training 


22  The  actual  patterns  beccnne  much  more  complicated  because  students  who  team  faster  or  better  are 
forced  to  wait  for  their  fellow  students  during  each  class  or  section  of  course  material.  In  lock-step 
courses,  this  extra  time  may  be  used  by  those  students  to  better  incorporate  the  knowledge  they  are 
gaining  into  their  understanding  of  the  tasks  to  be  trained  through  further  study  by  tutoring  their 
colleagues,  but  it  may  also  be  wasted.  Even  if  these  fast  learners  are  successfully  used  as  tutrws,  it  is 
iKX  clear  that  they  are  cost-effectively  used  as  tuttns.  This  reuims  to  the  previous  discussion  about 
managing  "fiee"  resources. 
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Providing  this  conceptual  framework  allows  us  insight  into  several  policy  issues. 
For  example,  it  is  possible  that  students  will  acquire  knowledge  differently  under  self- 
paced  or  group-paced  ICW  than  under  conventional  methods-^?  The  question  could  then 
be  asked:  Is  the  current  requirement  for  training  the  correct  requirement  under  ICW? 
Quantifying  this  conceptual  framework  can  aid  in  addressing  this  question.^'* 

Another  policy  issue  is  the  question  of  producing  more  than  minimally  qualified 
trainees.  Some  argue  that  self-pacing  and  testing  to  criteria  mean  that  those  who  would 
eventually  become  "A”  students  never  do  so.  because  they  have  not  had  the  extra  time 
imposed  on  them  by  being  lock-stepped  with  colleagues  who  do  not  learn  as  fast. 

However,  this  is  a  management  issue.  It  is  certainly  possible  to  allow  students  who 
progress  more  rapidly  to  cover  greater  amounts  of  material,  a  process  that  trades  the  benefit 
(cost  avoidance)  of  student  time  savings  for  the  benefit  of  greater  knowledge;  quantifying 
these  MP  curves  for  individual  students  could  help  guide  that  decision.25 

One  advantage  of  ICW  instruction  is  that  it  will  automatically  provide  the 
information  necessary  to  construct  these  MP  curves,  and  lock-step  instruction  does  not.  In 
general,  individual  MP  curves  would  give  us  a  better  understanding  of  the  meaning  of  the 
optimal  amount  of  training  for  each  individual. 


For  an  example  of  possible  differences  between  group  and  individualized  instruction,  see  Bloom  [291. 
He  found  that  students  taught  in  conventional  classroom  settings  scored  significantly  lower  on  standard 
tests  relative  to  students  given  the  same  amount  of  insuuctional  time  under  individual  tutoring.  This 
result  led  him  to  say  that  a  key  educational  challenge  is  to  find  instructional  methods  that  give  the 
effect  oi  individual  tutoring  without  the  cost.  ICW  may  be  one  such  method. 

24  It  may  be  that  the  change  in  the  proportion  of  Navy  courses  from  self-paced  to  lock-step,  mentioned  in 
the  discussion  of  Table  6  (Section  II.C.),  is  a  namely  occurring  experiment  that  would  allow  such  an 
estiinate  to  be  developed. 

23  For  example,  students  represented  by  MP  curves  such  as  MPi  could  be  allowed  to  continue  training 
until  the  value  of  that  additional  training  equals  its  cost 
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V.  SUMMARY  OF  FINDINGS  AND  RECOMMENDATIONS 


A.  SUMMARY  OF  FINDINGS 

At  any  given  time,  one  of  every  seven  active  duty  personnel  is  involved  in 
receiving,  giving,  or  supporting  individual  training.  This  activity  consumes  6.7%  of  tlie 
annual  DoD  budget.  Due  to  the  increase  in  both  the  number  of  material  systems  and  their 
complexity,  there  is  every  reason  to  believe  that  this  training  will  increase  in  importance 
with  respect  to  resources  consumed  and  to  achieving  needed  levels  of  readiness  and 
warfighting  capability.  Due  to  possible  changes  in  Total  Force  policy,  such  as  those  that 
increase  the  use  of  Reserve  components,  instructional  delivery  methods  that  can  be  more 
decentralized  and  available  on  student  demand  may  become  more  important 

DoD  di’wtives  and  instructions  exist  that  encourage  the  development  and  use  of 
innovative  technologies  for  training.  One  such  technology  is  interactive  courseware 
(ICW).  A  new  DoD  instruction  tries  to  develop  procedures  to  make  it  less  costly  to: 
maintain  and  enhance  ICW  material,  find  out  about  its  existence,  and  disseminate  it  to 
interested  parties. 

The  evidence  is  sparse,  but  ICW  does  not  appear  to  be  widely  used  in  military 
training.  Many  explanations  are  offered  for  its  lack  of  use.  One  possibility  is  that  the  costs 
avoided  for  individual  course  components  do  not  translate  into  savings  for  course 
sequences.  ICW  may  be  under-used  because  training  policymakers  are  not  aware  of  the 
full  costs  of  training  or  because  training  raanageis  are  not  given  incentives  to  manage  based 
on  those  costs.  Also,  there  may  be  qualitative  benefits  that  are  perceived  to  result  only 
from  group  pacing  and  instructor  mediation. 

A  large  body  of  studies  on  the  cost-effectiveness  of  ICW  has  reported  favorable 
results.  There  are  significant  time  savings  by  students  under  instruction,  a  savings  usually 
estimated  at  around  30%  for  individual  courses.  The  development  and  maintenance  costs 
for  thes3  ICW  technologies  seem  to  be  equal  to  or  less  than  conventional  instructional 
approaches.  Given  the  forecast  price  declines  in  the  hardware  and  software  used  for  ICW, 
there  is  every  reason  to  assume  its  relative  cost  will  fall  further. 
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A  rough  estimate  of  the  effect  of  applying  these  technologies  wnicly  in  individual, 
specialized  skill  training  suggests  annual  cost  savings  of  SLH)  miUion-Slf>0  million  and 
student  personnel  savings  of  9-1  i  thousand. 

Increased  use  of  ICW  will  tend  to  increase  the  amount  of  individualization, 
including  self-pacing,  in  training.  This  has  the  potential  to  change  the  way  students  Icam 
and  the  amount  of  knowledge  they  accumulate.  If  these  changes  are  sigmficani.  they  may 
change  the  amount  and  kinds  of  knowledge  and  skills  we  want  to  provide  through  training, 

B.  RECOMMENDATIONS 

The  data  and  analyses  presented  here  may  not  be  enough  to  motivate  a  complete 
change  in  policy  with  respect  to  the  insertion  of  ICW  into  military  training.  However, 
several  options  can  be  developed  based  on  the  material  presented; 

•  Because  ICW  training  appears  to  be  cost-effective,  efforts  should  be  made  to 
develop  policy  direction  that  would  make  consideration  of  such  technologies 
closer  to  the  default  approach  during  major  revisions  of  in.siruciionai  design 
and  delivery  methods. 

Our  examination  of  this  problem,  combined  with  conversations  with  training 
managers,  suggest  several  initiatives  that  might  give  incentives  for  greater  use  of  ICW. 
One  approach  would  be  closer  oversight  of  major  overhauls  of  course  material.  This  is  the 
place  where  new  requirements,  design,  and  delivery  approaches  will  be  analyzed  in  any 
case,  and  where  investment  in  courses  will  be  made.  It  seems  to  be  an  appropriate  place  to 
inject  policy  guidance  towards  investment  in  ICW  technology,  because  of  the  cost- 
effectiveness  improvements  offered  by  ICW. 

Particular  attention  should  be  paid  to  high  throughput  courses  and  course 
sequences.  If  it  can  be  shown  that,  for  example.  of  the  training  load  passes  thro  jgh 
5%  of  the  courses,  then  this  suggests  concentrating  the  use  of  ICW  on  those  courses. 
Such  an  example  from  Navy  data  is  shown  in  Appendix  A. 

One  way  to  implement  this  approach  would  be  to  make  ICW  the  rule  rather  than  the 
exception.  Course  developers  would  have  to  explain  why  they  have  chosen  not  to  use 
ICW  through  the  presentation  of  cost  and  effectiveness  data.  There  are  courses  in  which 
use  of  ICW  is  not  appropriate,  and  exceptions  could  easily  be  justified.  On  the  other  hand. 
Oils  requirement  would  to  some  degree  ensure  that  ICW  is  at  least  considered  in  DoD 
training  development. 


34 


Another  approach  would  involve  creating  a  small  budget  item  containing  funds  to 
be  competed  for  by  proposals  to  apply  ICW  technology.  This  would  give  incentives  for 
individual  initiative  by  training  organizations  that  have  evaluated  ICW  as  a  promising 
training  technology,  but  have  been  unable  to  generate  the  budget  necessary  to  undertake  the 
necessary  investment. 

Whatever  policy  is  suggested  should  be  sensitive  to  the  qualitative  issues  raised  in 
Section  rV.A.2. 

•  Cost  to  train  estimates  should  be  developed  for  all  ind-vidual  courses. 
Estimates  such  as  those  being  developed  under  the  initiative  on  unit  cost  to 
train  might  include  an  individual  incremental  cost  to  train,  as  well  as  an 
allocation  of  course  development  and  BOS  costs.  ^6  Where  possible,  these 
data  should  have  some  direct,  quantifiable  relationship  to  the  budget. 
However,  even  roughly  right  estimates  would  be  useful  for  policymaking 
purposes. 

These  estimates  should  be  used  to  measure  the  cost  of  alternative  training 
approaches.  In  this  way  training  managers  will  have  better  information  on  which  to  make 
cost-effective  training  decisions,  for  example,  in  response  to  the  policy  direction  in  the  first 
recommendation. 

•  Provisions  in  the  instruction  on  ICW  (DoD  Instruction  1322.20)  should  be 
applied  to  enhance  cost-effective  life  cycle  maintenance,  improvement,  and 
dissemination  of  ICW  technology. 

•  A  key  question  that  is  not  fully  answered  here  is  the  extent  to  which  ICW  and 
the  self-pacing  it  allows  are  being  utilized  in  current  training.  It  is  hard  to 
evaluate  current  practice  if  you  do  not  know  what  it  is.  Therefore,  rigorous 
quantification  of  the  use  of  ICW  and  self-pacing  technologies  in  training  for  all 
services  is  needed. 

•  Studies  should  be  done  to  develop  information  on  the  incremental  accumulation 
of  knowledge  under  different  instructional  delivery  systems  to  gain  better 
insights  on  whether  the  current  approach  of  training  to  specific  requirements 
should  be  modified  for  these  newer  approaches.  The  desirability  of 
encouraging  quick  learners  to  learn  more  under  self-paced  instruction  should 
be  investigated. 

•  The  effects  of  policy  recommendations  with  respect  to  other  aspects  of  training 
on  the  cost-effectiveness  of  ICW  should  be  monitored,  and  visa  versa.  In 


For  an  example  of  a  cost  taxonomy  that  would  be  more  than  sufficknt  for  this  and  many  other  costing 
exercises,  see  Kniq)p  and  Orlansky  [30].  BOS  costs  have  been  found  to  be  influenced  by  wwkload 
(e.g.,  Levine  et  al.  [31]). 
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particular,  one  of  the  assumptions  maintained  through  this  analysis  was  that  the 
timing  of  training  would  be  essentially  unchanged  (see  Section  III.B). 
However,  the  cost-effectiveness  of  ICW  might  have  an  effect  on  the  optimal 
timing  of  training,  and  changes  in  timing  of  training  could  have  effects  on  the 
cost-effectiveness  of  instructional  design  and  delivery  methods.  (For  more  on 
the  timing  of  training,  see  Wilson  and  Horowitz  [14]). 
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DISTRIBUTION  OF  TRAINING  LOADS  AND  ESTIMATES  OF 
ICW  SAVINGS  FOR  NAVY  INITIAL-SKILL  TRAINING 

INTRODUCTION 

The  cost  savings  reported  in  this  paper  occur  primarily  because  self-pacing  results 
in  less  time  spent  under  instruction  by  students,  producing  personnel  savings  by  reducing 
both  student  and  instructor  time.  These  savings  have  not  been  realized,  in  part  because 
service  training  establishments  do  not  have  the  incentives  to  convert  to  ICW.  A  key 
recommendation  of  this  report  is  to  develop  tl^se  incentives. 

One  issue  that  needs  to  be  understood  as  we  try  to  encourage  the  insertion  of  ICW 
into  military  training  is  determining  the  need  for  internal  or  external  incentives  to  undertake 
this  change.  This  need  depends  on  the  number  of  courses  and/or  pipelines  that  would  have 
to  be  modified  to  convert  20-25%  of  the  load.  If  the  number  of  courses  that  must  be 
converted  is  large,  then  a  system  must  be  developed  that  gives  individual  training  managers 
incentives  for  ICW  conversion  without  external  direction.  However,  if  the  number  of 
courses  and  pipelines  to  be  converted  is  small,  then  a  sophisticated  structure  that  relies  on 
internally-generated  incentives  is  not  necessary.  The  services  can  simply  direct  their 
training  establishments  to  either  convert  courses  or  pipelines  in  order  to  get  these  savings, 
or  to  provide  data  that  demonstrate  that  these  conversions  are  not  cost-effective. 

SUMMARY 

This  appendix  reports  the  distribution  of  student  load  over  the  courses  that  make  up 
Navy  &iUsted  training.  Thirty-one  percent  of  the  total  Navy  enlisted  student  load  in  1989 
(13,398  of  43,1 19)  was  concentrated  in  27  identifiable  courses.  One  of  those  courses  is 
already  converted  to  self-paced  ICW  and  several  of  the  other  courses  are  non-self-paced 
ICW.  The  estimated  20-25%  savings  through  changing  the  training  delivery  method  of 
these  relatively  few  courses  of  instruction  appear  to  be  reasonable. 

These  27  courses  involve  33  different  ratings  (Navy  enlisted  occupations),  or  44 
different  instructional  pipelines  (including  six-year  initial  obligations  and  nuclear  field 
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specialties).  Considering  the  entire  pipeline  is  important  if  student  time  savings  due  to 
rapid  completion  of  an  earlier,  self-paced  course  are  not  to  be  lost  while  waiting  for  a  later 
group-paced  course  to  begin.  By  including  the  other  courses  that  are  part  of  these 
pipelines,  we  raise  the  identifiable  courses  to  be  converted  to  72,  still  only  2.6%  of  the  total 
courses  of  instruction  in  Navy  enlisted  training  that  had  student  throughput  in  1989.  While 
a  few  specific  rating  pipelines  included  here  may  be  poor  candidates  for  conversion  to  self- 
paced  ICW,  the  basic  estimates  seem  to  hold. 

Navy  student  Pay  and  Allowances  was  40%  of  the  $2,108.4  million  on  which  the 
savings  estimate  of  $130  million  to  $160  million  was  based.  Therefore,  the  Navy  portion 
of  those  savings  would  be  $51  million  to  $63  million.  The  personnel  savings  are  in  the 
range  of  1,800  to  2,750. 

The  Navy  should  be  directed  to  convert  as  many  of  these  courses  as  possible,  or 
provide  new  information  that  explains  why  such  conversion  should  not  occur.  If  these 
results  hold,  the  other  services  should  also  be  directed  to  convert  to  ICW  all  courses  and 
associated  pipelines  with  more  than  a  specified  student  load  or  explain  why  conversion 
would  not  be  cost-effective. 

The  other  services  were  queried  for  data,  but  their  course  data  were  not  available  in 
the  form  needed  for  this  analysis.  While  conversations  with  subject  matter  experts  indicate 
that  these  savings  are  possible  in  the  other  services,  this  cannot  yet  be  documented. 

The  rest  of  this  appendix  discusses  the  analysis  that  is  summarized  above.  It 
reports  the  Navy  data,  and  the  definitions  used  to  distinguish  between  various  kinds  of 
courses.  Additionally,  it  describes  how  these  data  were  pared  down  to  reveal  the  relatively 
small  number  of  courses  that  contain  a  large  percentage  of  annual  student  load.  Finally,  the 
proportion  of  the  savings  attributable  to  Navy  courses  is  calculated.  The  personnel  and 
cost  results  are  summarized,  and  conclusions  and  recommendations  are  presented. 

DATA  AND  DEFINITIONS 

The  data  used  in  this  analysis  are  Navy  data  taken  from  the  Navy  Integrated 
Training  Resource  Analysis  System  (NTTRAS).  People  in  different  ratings  take  different 
course  sequences  to  order  to  obtain  the  training  required.  This  section  contains  three 
subsections.  The  first  subsection  gives  an  example  of  the  NTTRAS  data  available  for  a 
given  course  for  a  particular  rating.  The  second  describes  two  types  of  course  identifiers 
that  are  attached  to  each  course  and  discusses  their  importance  to  the  analysis.  The  third 
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subsection  shows  examples  of  how  one  or  more  courses  fit  together  to  provide  the 
instructional  sequence  for  one  rating. 

Data  for  a  Given  Course 

An  example  of  the  data  for  one  particular  course  for  one  particular  rating  is 
presented  in  Table  A- 1. 


Tabla  A*1.  SampU  Racord  Extractad  from  NiTRAS 


ON 

CDP 

Method  of 
Instruction 

D:^  Under 
Insuuction 

UIAOB 

AP 

A-06(MX)11 

61 IJ 

FTG-SS 

L 

29 

32.4 

Type  Course  indicators  starting  with  A  show  initial-skill  training  courses.  There 
are  two  different  course  identifiers,  CIN  (Course  IdentiAcation  Number)  and  CDP  (Course 
Data  Processing)  code,  whose  defmitions  will  be  discussed  below.  Pipeline  shows  a 
particular  rating  that  requires  this  course,  in  this  case  a  Submarine  Guided  Missile  Fire 
Control  Technician.  Method  of  Instruction  has  four  possible  values  depending  on  whether 
the  course  is  group-  or  self-paced,  and  whether  the  course  material  is  primarily  provided  by 
instructors  or  some  form  of  ICW.  In  this  case,  L  is  group-paced,  instructor-managed. 

The  number  of  days  actually  under  instruction  is  29,  or  one  day  under  six  weeks. 
(Another  length  indicator  is  Course  Length,  which  includes  "internal"  weekends.  Hence, 
10  instructional  days  (starting  on  a  Monday)  means  a  course  length  of  12  days,  11 
instructional  days  means  a  course  length  of  15.)  UI  AOB  stands  for  Under  Instruction 
Average-on-Board.  It  means  that,  on  an  average  day,  there  were  32.4  FTG-SS  trainees 
taking  CDP  61 IJ.  Assuming  there  are  approximately  240  days  actually  devoted  to 
instruction  in  one  year,  a  rough  estimate  of  the  throughput  of  this  part  of  the  FTG-SS 
pipeline  is  268  [(240/29)  x  32.4].i 

Two  Types  of  Coarse  Identifiers 

The  two  key  indicators  that  identify  individual  courses  are  the  CIN  and  CDP  code. 
The  best  way  to  give  a  flavor  for  the  difference  between  these  two  indicators  is  with  an 
example  from  the  civilian  educational  world.  The  CIN  is  the  equivalent  of  identifying 


^  We  did  not  ask  for  die  unnber  of  graduates  in  a  given  year  in  this  data  r^piesL 
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ECON  101  as  taught  on  all  campuses  of  the  State  University  of  New  York  (SUNY).  The 
CDP  would  be  the  equivalent  of  identifying  the  seats  in  section  3  of  ECON  101  as  taught  at 
the  Stony  Brook  campus  of  SUNY  that  are  set  aside  for  business  majors  (there  are 
occasional  exceptions  to  the  "particular  major"  part  of  this  analogy).  Therefore,  there  may 
be  multiple  CDPs  that  all  use  some  or  all  of  the  course  material  from  one  CIN. 

Both  the  CIN  and  the  CDP  indicators  are  important  in  this  analysis.  The  CINs  help 
identify  the  number  of  large  student  load  courses  that  could  be  converted  to  ICW. 
However,  the  more  different  CDPs  in  each  CIN,  the  more  different  instructional  pipelines 
are  likely  to  be  affected  by  converting  a  given  CIN  to  ICW.  A  mixture  of  self-paced  and 
group-paced  courses  in  a  pipeline  might  result  in  individuals  rapidly  finishing  self-paced 
courses,  only  to  have  to  wait  for  group-paced  courses  to  begin,  and  it  is  just  this  sort  of 
pipeline  management  that  is  reported  as  difficult  by  training  managers. 

Sample  Instructional  Pipelines^ 

Many  instructional  pipelines  are  extremely  simple.  For  example,  in  1989,  after 
recruit  training,  enlisted  personnel  who  are  to  become  Operations  Specialists  (OSs)  took 
one  course  at  one  location,  Operations  Specialist  A  school;  enlisted  personnel  in 
construction  ratings  each  take  their  own  single  A  school  course,  which  may  be  taught  at 
one  of  two  locations.  An  example  of  a  more  complicated  pipeline  is  Engineman  (EN).  In 
1989,  enlisted  personnel  training  for  this  rating  took  three  courses  at  one  location: 
Propulsion  Engineering  Indoctrination  and  Propulsion  Engineering  Basic  (both  of  which 
ate  given  to  several  different  ratings),  followed  by  Engineman  A  School.  Additionally, 
ENs  who  will  serve  on  submarines  then  take  Submarine  Service  Enlisted  Basic  at  another 
location. 

An  example  of  a  complicated  instructional  pipeline  is  Surface  Sonar  Technician 
(STG).  STGs  take  four  courses,  the  second  of  which  can  be  any  one  of  five  different 
courses,  each  of  which  may  be  a  different  length. 

ANALYSES  AND  RESULTS 

Two  analyses  to  identify  high  AOB  CINs  arc  presented  below.  The  first  analysis  is 
narrowly  defined,  looking  only  at  the  large  CINs,  and  ignoring  the  possibility  that  there 
would  be  interactions  between  these  courses  and  the  other  courses  in  given  instructional 
pipelines.  The  second  analysis  adds  in  the  additional  courses  that  might  have  to  be 


^  For  a  ccmplete  list  of  initial-skill  training  pipelines,  see  [A-1]. 
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converted  to  gain  full  ICW  benefits  for  instructional  pipelines  containing  high-volume 
CINs.  This  second  analysis  indicates  the  effort  needed  to  remove  the  problem  of  managing 
students  through  an  instructional  sequence  that  includes  group-  and  self-paced  courses. 
Managing  student  throughput  in  mixed  pipelines  is  one  of  the  primary  arguments  given  as 
to  why  it  is  not  worthwhile  to  convert  particular  courses  in  a  pipeline  to  ICW. 

Before  proceeding  with  these  analyses,  the  particular  subset  of  data  to  be  further 
analyzed  is  identiHed.  Next,  each  analysis  is  discussed  in  turn.  For  each  analysis,  the 
description  of  the  data  rearrangements  and  transformations  needed  to  identify  and  extract 
the  high-throughput  courses  is  reported.  The  student  load  for  these  large  courses  or  large 
instructional  pipelines  is  listed  and  compared  with  the  total  student  load. 

Summary  Data 

Data  that  summarize  the  number  and  student  load  in  Navy  formal  school  training 
courses  are  shown  in  Section  II.C,  Tables  4  and  S.  The  data  subject  to  further  analysis  will 
be  the  courses  and  load  in  the  initial-skill  training  type  course.  These  data  indicate  that 
there  were  314  CDPs  that  had  non-zero  UI AOB  in  FY  1989.  The  UI AOB  load  in  those 
courses  was  23,048,  or  53.5%  or  the  total  UI  AOB  for  the  year.  The  reasons  for 
restricting  the  analysis  to  these  courses  are:  (1)  that  these  tend  to  be  the  courses  with  the 
largest  student  load,  and  (2)  the  instructional  pipelines — the  relationships  between 
courses — are  most  explicit 

Identification  of  Likely  Conversion  Candidates 

This  subsection  describes  the  data  we  extracted  to  yield  the  27  courses  mentioned  in 
the  summary.  Basically,  the  selection  was  based  on  two  criteria:  (1)  how  certain  are  we  a 
course  is  suitable  for  conversion  into  self-paced,  computer-assisted  instruction,  and  (2)  the 
size  of  the  course  as  measured  by  UI  AOB.  With  respect  to  the  first  criterion,  the  more  we 
are  likely  to  know  about  how  a  course  fits  into  an  overall  instructional  pipeline,  and  the 
more  we  believe  it  covers  a  definite  body  of  tasks  and  skills  knowledge,  the  more  likely  we 
were  to  select  it 

The  second  criterion  is  used  because  conversion  to  ICW  has  some  fixed  costs.  If  a 
large  course  is  converted,  there  are  more  instructors  and  equipment  saved  to  offset  the  fixed 
costs  of  conversion.  Also,  the  larger  the  throughput,  the  more  student  time  will  be  saved 
from  the  expenditure  of  that  given  conversion  cost 

The  number  of  CENs  and  CDPs  that  were  selected  and  not  selected  are  shown  in 
Table  A-2.  First,  we  concentrated  only  on  the  initial-skill  courses.  Many  advanced 
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courses  have  smaller  class  sires,  and  it  is  less  likely  that  they  must  be  taken  in  a  particular 
sequence.  However,  a  further  examination  of  these  courses  could  be  undertaken  to  see  if 
there  are  a  few  additional  large  courses  that  could  be  converted,  because  greater  pipeline 
flexibility  and  more  experienced  students  may  make  ICW  a  more  effective  training 
approach. 


Table  A-2.  CIN,  COP,  and  Ul  AOB  Data  Extraction 


Category 

Initial 

No  UIAOB 

No  UIAOB 

Sub  1 

Igncxed 

Sub  2 

Small 

Sub  3 

ON 

235 

13 

0 

222 

98 

124 

97 

27 

CDP 

357 

27 

16 

314 

122 

192 

121 

71 

UIAOB 

23,048 

23,048 

4,212 

18.836 

4,938 

13,398 

The  first  cut  occurs  because  no  one  took  some  courses.  In  the  initial-skill  group 
there  were  357  CDPs  that  are  part  of  235  CINs.  In  FY  1989,  there  were  13  CINs  (27 
CDPs)  with  no  UI  AOB,  and  16  CDPs  with  no  UI  AOB,  even  though  other  CDPs  in  the 
same  CIN  had  UI  AOB.  Therefore,  there  were  222  initial-skill  training  CINs,  making  up 
314  CDPs,  with  23,048  UI  AOB  in  FY  89  as  shown  under  Sub  1  in  Table  A-2. 

The  second  cut  keys  on  our  uncertainty  about  whether  the  material  is  suitable  to 
generate  student  time  savings.  It  includes  courses  that  were  not  attached  to  particular  rating 
pipelines.  The  latter  category  includes  Apprenticeship  training  (Airman,  Seaman,  Fireman) 
and  some  courses  that  include  officers.  The  first  part  of  this  cut  is  a  measure  of  our 
uncertainty  about  the  courses  themselves.  It  may  be  that  these  courses  are  actually  easier  to 
convert  to  ICTW,  but  it  is  difficult  to  know  this  for  certain. 

The  Apprenticeship  training  is  removed  because  the  material  being  presented  here 
may  have  a  much  stronger  component  of  military  practice  and  team-building  than  of  rating- 
specific  tasks,  skills,  and  knowledge.  However,  these  are  large  courses  and  could  be 
reviewed  in  more  detail  for  potential  conversion. 

This  second  cut  removed  98  CINs  constituting  122  CDPs,  and  4,212  UI  AOB.  and 
left  124  CINs.  192  CDPs,  and  18,836  UI  AOB  as  shown  in  Sub  2  in  Table  A-2. 

The  third  cut  occurs  due  to  class  size.  Seventy  one  percent  of  the  remaining  UI 
AOB  is  in  22%  of  the  CINs.  This  removes  another  97  CINs,  121  CDPs,  and  4,938  UI 
AOB,  as  shown  in  Sub  3  of  Table  A-2,  The  remaining  27  CINs  are  listed  in  Table  A-3. 
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Table  A-3.  Remaining  Large  Courses 


Type 

Course 

Course 

Days  Under 
Instruction 

UIAOB 

Indi¬ 

cator 

Group-Raced,  Instructor-Managed 

A1 

Electronics  Tedmidan  A  Sdiool,  Phase  II 

140 

854.0 

A1 

Rib  Control  Technician  A  School 

no 

790.7 

A1 

Operations  Specialist  A  School 

70 

750.6 

I 

A1 

Machinist’s  Mate  Nuclear  Field  A  School 

60 

587.2 

AP 

Submarine  Service  Enlisted  Basic 

29 

464.4 

M 

A1 

Data  Systmns  Technician  A  School 

160 

421.2 

*1 

A1 

Electronics  Technician  Nuclear  Field  A  School 

135 

412.4 

A1 

Electrician’s  Mate  Nudear  Field  A  School 

75 

347.2 

A1 

Basic  Electronics  Rate  Training 

70 

249.7 

A1 

Music  Basic 

119 

217.5 

1 

A1 

Gas  Turbine  Systems  Technician  Mechanical  A  School 

68 

212.5 

A1 

Gunner’s  Mate  A  School  Phase  I 

120 

211.1 

Subtotal 

5,518.5 

Self-Paced,  Computer-Managed 

AP 

Basic  Electridty  and  Electronics 

55 

770.9 

M 

Group-Raoed,  Computer-Assisted 

A1 

Avionics  Technician  A  School 

111 

1,584.0 

1 

A1 

Aviation  Electrician’s  Mate  A  School 

110 

715.0 

1 

A1 

Radicnnan  A  School 

65 

583.2 

1 

A1 

Electronics  Technician  A  School  Phase  I 

85 

574.9 

At 

Engineman  A  School 

105 

523.0 

A1 

Interior  Ccmununications  Technician  A  School 

no 

518.6 

*1 

AP 

Rropulsion  Engineering  Basic 

23 

486.4 

M 

A1 

Mess  Management  Specialist  A  School 

35 

469.6 

*1 

A1 

Machinist’s  Mate  A  School 

42 

362.5 

A1 

Aviation  Oidnanceman  A  School 

52 

317.7 

1 

A1 

Basic  Non-Motse  Operations 

50 

262.0 

A1 

Aviation  Machinist’s  Mate  A  School 

42 

256.9 

1 

A1 

Aviatitm  Structural  Mechanic  -  Structures  A  School 

46 

233.6 

1 

A1 

Air  Traffic  Controller’s  A  School 

80 

221.1 

1 

Subtotal 

7,108.5 

Total 

13,397.9 

Notes:  Tbe  ’’indicator’’  column  refers  to  the  kind  of  pipeline  or  pipelines  with  which  a  course  is  associated;  i 
shows  exactly  one  course  in  tbe  pipeline,  *l  shows  one  course  plus  Submarine  Service  Enlisted  Basic  for 
enlisted  submariners,  and  M  shows  the  courses  that  are  given  to  many  different  ratings.  Tbe  remaining 
courses  are  either  taught  to  a  few  ratings,  part  of  multicourse  sequences,  or  both. 


These  27  CINs  include  31%  of  the  total  UI AOB  for  the  Navy  in  1989  (13,398  of 
43,119).  These  27  CINs  are  only  1%  of  the  total  CINs  (2,787)  that  had  UI  AOB  in  1989. 
One  of  these  courses  (Basic  Electricity  and  Electronics)  is  already  self-paced  and  computer- 
managed,  and  53%  of  the  UI  AOB  for  these  27  CINs  is  group-paced  and  computer- 
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assistsd.  This  means  that  these  group-paced,  computer-assisted  courses  have  been  partially 
or  completely  converted  to  ICW,  but  are  being  taught  in  a  way  that  probably  does  not  allow 
all  the  student  time  savings  to  be  fully  realized. 

Even  if  only  half  the  potential  ICW  student  time  savings  is  still  available,  convening 
12  CINs  from  group-paced,  instructor-managed  to  self-paced  ICW,  and  14  CINs  from 
group-paced  ICW  to  self-paced  ICW  would  meet  the  20%  of  student  load  coverage 
requirement  (5,518.5  +  (7,108.5/2)  =  9,173,  or  21%).  Also,  conversion  costs  for  the 
group-paced,  computer-assisted  courses  should  be  much  lower,  since  much  of  the  work  is 
already  done.^ 

One  way  to  understand  the  effort  necessary  to  make  this  change  is  to  look  at  the 
total  number  of  days  of  coursework  that  would  have  to  be  converted  from  lock-step  to  self- 
paced.  The  sum  of  days  under  instruction  for  the  12  lock-step  courses  above  is  1,156,  or 
4.8  instructional  years  using  240  days  per  year.  The  sources  cited  in  the  main  text  repon 
that  these  conversion  costs  will  be  approximately  equal  to  or  less  than  the  savings  in 
instructor  pay  and  allowances  and  other  operations  and  maintenance  cost  savings  derived 
from  the  conversion. 

Conversion  of  the  Rest  of  the  Affected  Pipelines 

Section  IV. A.  1  reports  concerns  of  training  managers  that  the  full  benefits  of  ICW 
would  not  be  available  in  course  sequences  where  some  of  the  courses  were  self-paced  and 
others  were  group-paced.  That  is,  student  time  under  instruction  might  be  saved  in  a  self- 
paced  course,  but  wasted  as  early  completers  wait  for  lock-step  courses  to  begin.  Such 
concerns  need  to  be  considered  here,  because  these  27  CINs  and  7 1  CDPs  cover  33 
different  ratings  and  44  different  pipelines.  This  coverage  means  that,  after  these  large 
throughput  courses  are  modified  through  the  inclusion  of  ICW  technology,  either  care  must 
be  taken  to  integrate  them  with  the  non-ICW’  courses  that  make  up  the  rest  of  the  pipelines, 
or  the  rest  of  the  pipelines  need  to  be  converted  to  ICW, 

In  order  to  understand  the  magnitude  of  this  problem,  the  CDPs  were  sorted  by 
rating,  and  all  CDPs  were  retained  in  any  rating  that  had  one  of  the  large  courses. 
Additionally,  if  a  new  CDP/CIN  was  retained,  any  rating  that  used  that  course  was  also 
included.  The  effects  of  this  addition  are  shown  in  Table  A-4,  which  replicates  much  of 


^  Infonnadon  in  OrUinsky  [A-2]  suggests  that  more  than  half  of  all  savings  in  such  conversions  comes 
from  self-pacing.  If  this  division  of  savings  applies  here,  the  estimate  we  have  constructed 
underestimates  the  potential  savings. 
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Table  A-2.  As  can  be  seen  from  the  column  labeled  “Adds."  the  average  UI  AOB  per  CLN 
for  those  courses  that  were  added  back  is  quite  small. 

TabI*  A-4.  Ravisad  CIN,  CDP,  and  UI  AOB  Data  Extraction 


Category 

initial 

NoUl 

AOB 

Sub  1 

Iffxxed 

Sub  2 

Small 

Sub  3 

Adtfe 

Sub  4 

ON 

235 

13 

222 

98 

124 

97 

27 

45 

72 

CDP 

357 

43 

314 

122 

192 

121 

71 

52 

123 

UIAOB 

23.048 

23.048 

4.212 

18.836 

4.938 

13.398 

2.647 

16.045 

Av.  AOB 

98 

104 

43 

152 

51 

496 

59 

223 

Nole:  Average  UI  AOB  ii  average  by  CIN. 


Examples  of  these  additions  are  Torpedoman's  Mate  (TM)  Submarine  Operator 
(SUB)  and  TM  Surface  Curator  (SUPE).  TM(SUB)  personnel  take  Submarine  School 
Enlisted  Basic,  one  of  the  27  large  CINs,  and  two  additional  CINs.  TM(SURF)  personnel 
lake  the  same  two  additional  CINs,  though  they  have  different  CDP  identifiers.  Therefore, 
these  two  ratings  contribute  two  CINs  and  four  CDPs  to  the  "Adds"  column. 

Additionally,  these  72  CINs  were  cumulated  by  method  of  instruction.  The  results 
are  shown  in  Table  A-5.  Once  again,  assuming  only  one-half  of  the  savings  in  the  group- 
paced,  computer- assisted  courses  are  still  available.  11,009  or  25.5%  (11,009  = 
(8,491.6/2)  -f  6,762.8)  of  the  UI  AOB  can  be  converted.  Looking  at  the  total  number  of 
lock-step  days  to  be  converted  yields  2,464.  or  10.3  student  years  of  material. 


Tabl«  A-S.  CINs  by  Msthod  of  Instruction 


Method  of  Instruction 

UIAOB 

Number  of  CINs 

Group-paced,  oomputer-assisied 

8.491.6 

32 

Self-paced,  computer-assisted 

790.3 

2 

Group-paced,  instructor-managed 

6.762.8 

38 

Total 

16.044.7 

72 

The  ratings  and  pipelines  included  in  the  last  column  of  Table  A-4  are  shown  in 
Table  A-6.  These  pipelines  are  grouped  by  general  areas.  Also,  the  pipelines  that  are 
included  as  a  result  of  the  broader  definition  aie  marked  with  an  asterisk.  As  can  be  easily 
seen,  many  of  the  pipelines  of  interest  under  either  approach  are  in  the  areas  of  aviation, 
electronics/radar/sonar,  and  propulsion/mechanics.  The  12  added  pipelines  fail  into  either 
the  AviatitHi  or  Other  category. 
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Tabitt  A-6.  Rating  Pipaiinas  Inciudad  in  Savings  Possibilitias 


Aviation 

Electromcs/Radar/Sonar 

Prooulsion/Mechanics 

Other 

AD 

ET 

m 

AC 

AE 

ET-AEF 

BT-6YO 

BASltEL* 

AME* 

ET-NF 

GM 

CTA* 

AMH* 

ET-SS 

GSE 

CTO* 

AMS 

GSE-6YO 

CTT 

AO 

RM 

GSM 

CTTT 

AQ 

RM(SS) 

GSM-6YO 

DS 

ACHSYO 

EN 

FC 

ASE* 

STG 

IC 

ASM* 

STS 

EM 

MS 

AT 

SWSELEC 

EM-NF 

OS 

AT-6YO 

MM 

PN* 

AV 

MM-6YO 

PR* 

AX 

AX-6YO 

MMN 

QM* 

QM{SS) 

SK 

TM(SUB) 

TM(SURF)* 

YN* 

Note;  See  Amiex  A  for  a  list  of  ratios  and  pipeline  names. 


NAVY  COSTS  AND  SAVINGS 

The  main  text  uses  cost  data  provided  in  the  MMTR,  by  the  Training  Performance 
and  Data  Center  (TPDC),  and  in  the  research  literature  to  develop  the  all-services  savings 
estimate  of  $130  million  $160  million.  This  subsection  briefly  summarizes  these  data  both 
in  the  text  and  in  Table  A-7,  and  shows  the  proportion  of  these  costs  that  are  attributable  to 
the  Navy. 

The  total  training  funding  reported  in  the  MMTR  for  1989  is  $19.4  billion,  $12.9 
billion  of  that  amount  are  direct  costs,  and  $6.0  billion  are  for  specialized-skill  training 
[A-3].  Of  the  amount  for  specialized-skill  training,  $2.1  billion  is  student  pay  and 
allowances.  It  is  assumed  that  25%  of  these  pay  and  allowances  can  be  saved  on  25%  of 
the  courses,  for  a  total  annual  savings  of  $132  million. 

The  $6.5  billion  difference  between  the  total  cost  and  direct  cost  is  made  up  of 
indirect  costs  of  training  and  the  cost  of  training  reserves.  That  amount  is  made  up  of  Base 
Operating  Support  (BOS)  costs.  Permanent  Change  of  Station  (PCS),  Temporary  Duty 
(TDY)  costs,  and  Reserve  component  pay  and  allowances.  Different  assumptions  are  made 
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about  the  proportion  of  these  costs  that  will  vary  with  changes  in  student  pay  and 
allowances.  (See  Table  10  in  the  main  text)  The  proportions  together  yield  an  estimate  of 
the  indirect  cost  that  varies  with  student  load.  The  total  amount  ($2  billion)  is  added  to 
student  pay  and  allowances  to  yield  $4.1  billion  as  an  estimate  of  the  total  cost  that  varies 
with  student  load.  Finally,  it  is  assumed  that  20%  of  the  courses  will  be  converted,  saving 
20%  of  the  cost  of  sending  students  through  the  converted  courses,  yielding  savings  of 
$164  million. 

Table  A-7.  All  Services  and  Navy-Only  Savings  Estimates  (1989  Doliars) 


Cateeories 

DoD  Estimates 
(Billions) 

Navy  Estimates 
(Billions) 

Total  Budget 

S19.4 

$5.8 

Direct  Costs 

12.9 

4.6 

Specialized-Skill  Training 

6.0 

2.0 

Active  Duty  Pay  and  Allowances 

2.1 

0.8 

25%  Tune,  25%  Cost 

.132 

.051 

Plus  Overhead  Allocation 

20%  Time,  20%  Cost 

.164 

.063 

In  1989,  Navy  student  pay  and  allowances  were  38.5%  ($810  million)  of  the  $2.1 
billion  student  pay  and  allowances.  Applying  that  percentage  to  the  $132  million  yields 
$51  million.  Assuming  the  same  indirect  cost  percentage  addition,  then  applying  that 
percentage  to  the  $164  million  yields  $63  million.*  The  sivings  of  student  personnel  in  the 
individuals  account  would  range  from  1,800  (20%  of  9,173)  to  2,750  (25%  of  1 1,009). 

SUMMARY 

For  Navy  enlisted  training,  conversion  of  courses  that  contain  20-25%  of  the 
student  load  appears  feasible.  The  20%  figure  can  be  attained  by  converting  12  lock- step 
CINs,  or  less  than  one-half  of  one  percent  of  all  CINs  that  had  UI  AOB  in  1989,  to 
computer-assisted,  self-paced  courses,  and  moving  14  group-paced,  computer-assisted 
CINs,  or  0.5%  of  all  UI  AOB  CINs  to  self-pacing  would  generate  savings  in  the  20% 
range,  approximately  1,800  personnel.  The  25%  figure  can  be  obtained  by  converting  an 


*  Once  again,  by  using  the  extreme  estimates  (20%  and  20%  on  Pay  and  Allowances  only  and  2S%  and 
25%  on  the  costs  that  include  ovetfaead  allocation)  the  range  of  the  estimates  could  be  increased,  in  this 
case  to  .^32  million  to  $98  million. 
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additional  26  lock-step  CINs,  (1%  of  all  CINs  that  had  UI  AOB  in  1989)  to  computer- 
assisted,  self-paced  courses.  Moving  an  additional  18  group-paced,  computer-assisted 
CINs  (0.7%  of  all  1989  CINs  with  UI  AOB)  to  self-pacing  would  generate  savings  in  the 
25%  range,  approximately  2,750  personnel.  Converting  the  additional  courses  to  self¬ 
pacing  means  entire  instructional  pipelines  become  self-paced,  decreasing  concerns  about 
problems  managing  flows  of  students  between  courses  in  a  pipeline. 

Results  of  these  analyses  are  encouraging.  Thirty  one  percent  of  all  Navy  UI  AOB 
is  contained  in  27  courses.  This  means  that  the  expense  of  developing  ICW  course  material 
that  covers  a  significant  part  of  the  total  student  throughput  should  be  relatively  low 
compared  to  the  total  amount  of  course  material.  In  addition,  approximately  half  of  this 
subset  of  AOB  is  in  14  CINs  that  are  already  partially  convened  to  ICW,  but  being  taught 
in  a  way  that  probably  does  not  allow  all  of  the  student  time  savings  to  be  fully  realized. 
Even  if  one-half  the  potential  ICW  student  time  savings  from  the  partially  converted  group 
are  already  being  realized,  the  other  half  are  still  available  to  be  saved.  Convening  these 
CINs  to  self-paced  ICW  would  meet  the  20%  of  student  load  coverage  estimate  developed 
in  the  main  text 

Even  including  conversion  of  all  courses  in  affected  pipelines,  the  results  are  still 
good.  Forty  percent  of  the  total  Navy  UI  AOB  is  contained  in  72  CINs  that  are  related  to 
the  rating  pipelines  that  contain  the  27  largest  CINs.  If  the  group-paced  courses  among  the 
additional  45  CINs  were  all  converted  to  self-paced,  it  would  increase  the  overall  cost  of 
conversion,  though  18  of  the  additional  courses  are  already  computer- assisted,  hence  they 
are  probably  already  achieving  some  of  the  cost  savings  from  self-pacing.  If  only  half  the 
savings  are  stUl  available  in  the  group-paced,  computer-assisted  courses,  this  still  results  in 
25%  of  the  training  load  being  convertible.  However,  because  the  throughput  in  the 
additional  ratings  is  small,  it  might  be  decided  not  to  convert  all  those  courses. 

CONCLUSIONS  AND  RECOMMENDATIONS 

This  analysis  demonstrates  that  the  student  load  in  Navy  enlisted  training  is 
concentrated  in  a  relatively  small  number  of  courses.  It  seems  desirable  to  direct  the  Navy 
to  convert  these  courses  to  self-paced  ICW  unless  comi>elling  counter-arguments  can  be 
made  in  specific  cases.  The  other  services  should  be  directed  to  provide  the  data  needed  to 
support  similar  analyses.  If  this  proves  impossible,  they  should  be  directed  to  convert  to 
self-paced  ICW  courses  or  instructional  pipelines  with  more  than  a  specified  student  load 
unless  they  can  provide  convincing  arguments  to  the  contrary.  This  direction  should  result 
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in  cost  savings,  less  time  spent  by  personnel  as  both  students  and  instructors  in  the  training 
establishment,  and  no  degradation  of  training  effectiveness. 
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ANNEX  A 


RATINGS  LIST 

ABE  (Aviation  Boatswain's  Mate  -  Equipment) 

ABF  (Aviation  Boatswain's  Mate  -  Fuels) 

ABH  (Aviation  Boatswain's  Mate  -  Handling) 

AC  (Air  Traffic  Controller) 

AD  (Aviation  Machinist's  Mate) 

AE  (Aviation  Electrician's  Mate) 

AFFR  (Aircraft  Fire  Fighting  and  Rescue) 

AG  (Aerographer's  Mate) 

AR  (Aviation  Storekeeper) 

AME  (Aviation  Structural  Mechanic  -  Safety  Equipment) 

AMH  (Aviation  Structural  Mechanic  -  Hydraulics) 

AMS  (Aviation  Structural  Mechanic  -  Structures) 

AO  (Aviation  Ordnanceman) 

AQ  (Avionics  Tech  -  Aviation  Control  Technician) 

AQ  (Avionics  Tech  -  Aviation  Control  Technician  -  6YO) 

ASE  (Aviation  Support  Equipment  Technician  -  Electrical) 

ASM  (Aviation  Support  Equipment  Technician  -  Mechanical) 

AT  (Avionics  Technician) 

AT  (Avionics  Technician  -  6YO) 

AV  Non-Navy  (Aviorucs  Technician) 

AW  (Aviation  Anti-Submarine  Warfare  Operator) 

AX  (Avionics  Technician  Aviation  Anti-Submarine  Warfare) 

AX  (Avionics  Technician  Aviation  Anti-Submarine  Warfare  -  6YO) 
AZ  (Aviation  Maintenance  Administration) 

BASHH^  (Basic  Helicopter) 

BT  (Boiler  Technician  PSI) 

BT  (Boiler  Technician  (PSI  -  6YO)) 

BU  (Builder) 

CE  (Construction  Electrician) 
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CM  (Construction  Mechanic 
CTA  (Cryptologic  Technician) 

Cn  (Cryptologic  Teclmician) 

CTM  (Cryptologic  Maintenance  Technician) 

CTO  (Cryptologic  Technician  O) 

CTR  (Cryptologic  Technician  R) 

CTT  (Cryptologic  Technician  T) 

CTT  (Cryptologic  Technician  Others) 

£>C  (Damage  Control 
DP  (Disbursing  Clerk) 

DP  (Data  Processing  Technician) 

DS  (Data  Systems  Technician) 

DT  (Dental  Technician) 

EA  (Engineering  Aid) 

EM  (Electricians  Mate) 

EM-NF  (Electrician's  Mate  Nuclear  Field) 

EN  (Engineman) 

EO  (Equipment  Operator) 

ET-AEF  (Electronics  Tecrmician  Advanced  Electronics 
ET  (Electronics  Technician  -  Otliers) 

ET-NF  (Electronics  Technician  Nuclear  Rdd) 

ET-SS  (Electronics  Technician  Submarine) 

EW  (Electronic  Warfare  Technician) 

PC  (Firecontrolman) 

FTG-SS  (Rre  Control  Technician  Underwater) 

GM  (Gunner’s  Mate) 

GSE  (Gas  Turbine  Systems  Technician  Elec) 

GSE  (Gas  Turbine  Systems  Technician  Elec  -  6YO) 
GSM  (Gas  Turbine  Systems  Technician  Mech) 

GSM  (Gas  Turbine  Systems  Technician  Mech  -  6YO) 
HM  (Hospitalman) 

HT  (Hull  Maintenance  Technician) 

IC  (Interior  Communications  Electrician) 

IM  (Instrumentman) 

IS  (Intelligence  Specialist) 

JO  (Information  Specialist  Journalist) 
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LI  (Lithographer) 

ML  (Molder) 

MM  (Machinist's  Mate) 

MM  (Machinist's  Mate  -  6YO) 

MMN  (Machinist's  Mate  Nuclear  Field) 

MN  (Mineman) 

MR  (MachinCTy  Repairman) 

MS  (Mess  Management  Specialist) 

MU  (Music  Basic) 

OM  (Qpticalman) 

OS  (Operations  Specialist) 

OTA  (Ocean  Systems  Technician  Analyst) 

OTM  (Ocean  Systems  Technician  Maintainer  -  6YO) 
PC  (Postal  Clerk) 

PH  (Photographer's  Mate) 

PM  (Pattern  Maker) 

PN  (Personnelman) 

PR  (Aircrew  Survival  Equipmentman) 

QM  ((Quartermaster) 

QM-SS 

RM  (Radioman) 

RM-SS  (Radioman  Submarines) 

RP  (Religious  Programs  Specialist) 

SH  (Ship's  Serviceman) 

SK  (Storekeeper) 

SM  (Signalman) 

STG  (Surface  Sonar  Technician) 

STS  (Sonar  Technician  -  Submarine) 

SW  (Steelworioer) 

SWS  Elec  (Strategic  Weapon  System  Electronics) 
TM  (Torpedoman's  Mate  Surface  Operator) 

TM  (Torpedoman's  Mate  Submarine  Operator) 

UT  (Utilidesman) 

WT  (Weapons) 

YN  (Yeoman) 
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ABBREVIATIONS 


ABBREVIATIONS 


6YO 

AOB 

BOS 

CAI 

CDP 

CIN 

CNET 

CPU 

DoD 

GUI 

ICW 

IDA 

MNfTR 

NTTRAS 

O&S 

OSUT 

PCS 

RAM 

TOY 

TPDC 

UI 


six-year  obligation 
Average-on-Board 
Base  Operating  Support 
computer-assisted  instruction 
Course  Data  Processing 
Course  Identification  Number 
Chief,  Naval  Education  and  Training 
central  processing  unit 
Department  of  Defense 
graphical  user  interface 
Interactive  Coursevtare 
Institute  for  Defense  Analyses 
"Military  Manpower  Training  Report" 

Navy  Integrated  Training  Resource  Analysis  System 

operating  and  support 

One  Station  Unit  Training 

Permanent  Change  of  Station 

random  access  memory 

Temporary  Duty 

Training  Performance  and  Data  Center 
Under  Instruction 
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